ai-forever / ai-forever/deepagents-gigachat

LoopBreaker: restore nudge repeatability (regression shipped in 0.0.4)

Ouverte
#7 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
16
Forks
3
Merge moyen
2 min
PR mergées (30 j)
1

Description

## Problem

0.0.4 changed nudge de-duplication in `LoopBreakerMiddleware`:

- **0.0.3** — `_already_nudged()` walks the history backwards and **breaks at the first `AIMessage`**. The marker is only visible in the last block, so the budget nudge is **re-injected on every model turn** while the condition holds.
- **0.0.4** — `_nudge_count()` counts markers across the whole history, so each nudge fires **once per trajectory** (this is deliberate in #6 and marked by a comment).

The effect is not "the order arrives later", it is "the order stops arriving". A long trajectory that used to be pushed to finish by a repeating instruction now drifts until it hits the graph recursion limit.

## Evidence

Both measured on harness-bench-fast v0.16.0, 391 tasks, k=4 per arm, arms interleaved on one host and one stand.

**GigaChat-3.5-432B-A28B (IFT), native `run` runner** — the plus from the deterministic-output gate outweighs this, but the loss is visible: 0.0.4 loses 31 long multi-file tasks (renames, module splits, migrations), and step-limit hits go 2.2% -> 4.0%.

**GigaChat-3.1-10B reasoning (`opd` step 200), `run-openrouter`** — here nothing offsets it, because the three workspace-dependent middlewares are inert under that runner (it never calls `set_workspace_path`):

| arm | mean / 391 | recursion-limit hits |
|---|---:|---:|
| 0.0.3 | 292.8 (74.9%), sd 4.4 | 2.7% |
| PR #6 | 251.0 (64.2%), sd 8.3 | 23.7% |

Losses grow with task horizon (median steps of that task's successful attempts on 0.0.3): 0-8 steps −3.3 pp, 9-14 −10.2, 15-22 −22.4, 23+ −25.7.

**The threshold numbers are not the cause.** A fourth arm took PR #6 whole and reverted only the constants (24 -> 12 and 30 -> 16): 255.0/391 (65.2%), sd 5.8 — indistinguishable from PR #6 at 251.0. The tell is the distribution of tool rounds, not the mean:

| arm | 0-3 | 4-7 | 8-11 | 12-15 | 16+ | hit 80-step limit |
|---|---:|---:|---:|---:|---:|---:|
| 0.0.3 | 417 | 559 | 220 | **311** | 14 | 43 (2.7%) |
| PR #6 | 412 | 597 | 174 | 9 | 2 | 370 (23.7%) |
| thresholds reverted | 389 | 609 | 175 | **7** | 1 | 383 (24.5%) |

At the same threshold of 12 the pile past the threshold did not come back. In 0.0.3, 325 of 1521 recorded attempts (21.4%) reach 12 rounds and 145 of them succeed — 12.4% of all successes of that arm live in the band where the hard order has already fired. In PR #6 that band holds 11 attempts and 7 successes.

## Suggested fix

Keep `_nudge_count` for the grep/error-loop nudges (capping them at 2 is reasonable), but let the **budget** nudge re-inject while its condition holds — that is the safety valve. Whatever the shape, it needs its own measurement: the combination "PR #6 gate + repeatable nudge" has never been run.

## Workaround until then

Pin `deepagents-gigachat==0.0.3` when running the profile against a weaker model.

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.