ai-forever / ai-forever/deepagents-gigachat
LoopBreaker: restore nudge repeatability (regression shipped in 0.0.4)
- Langage dominant
- Python
- Étoiles
- 16
- Forks
- 3
- Merge moyen
- 2 min
- PR mergées (30 j)
- 1
Description
## Problem
0.0.4 changed nudge de-duplication in `LoopBreakerMiddleware`:
- **0.0.3** — `_already_nudged()` walks the history backwards and **breaks at the first `AIMessage`**. The marker is only visible in the last block, so the budget nudge is **re-injected on every model turn** while the condition holds.
- **0.0.4** — `_nudge_count()` counts markers across the whole history, so each nudge fires **once per trajectory** (this is deliberate in #6 and marked by a comment).
The effect is not "the order arrives later", it is "the order stops arriving". A long trajectory that used to be pushed to finish by a repeating instruction now drifts until it hits the graph recursion limit.
## Evidence
Both measured on harness-bench-fast v0.16.0, 391 tasks, k=4 per arm, arms interleaved on one host and one stand.
**GigaChat-3.5-432B-A28B (IFT), native `run` runner** — the plus from the deterministic-output gate outweighs this, but the loss is visible: 0.0.4 loses 31 long multi-file tasks (renames, module splits, migrations), and step-limit hits go 2.2% -> 4.0%.
**GigaChat-3.1-10B reasoning (`opd` step 200), `run-openrouter`** — here nothing offsets it, because the three workspace-dependent middlewares are inert under that runner (it never calls `set_workspace_path`):
| arm | mean / 391 | recursion-limit hits |
|---|---:|---:|
| 0.0.3 | 292.8 (74.9%), sd 4.4 | 2.7% |
| PR #6 | 251.0 (64.2%), sd 8.3 | 23.7% |
Losses grow with task horizon (median steps of that task's successful attempts on 0.0.3): 0-8 steps −3.3 pp, 9-14 −10.2, 15-22 −22.4, 23+ −25.7.
**The threshold numbers are not the cause.** A fourth arm took PR #6 whole and reverted only the constants (24 -> 12 and 30 -> 16): 255.0/391 (65.2%), sd 5.8 — indistinguishable from PR #6 at 251.0. The tell is the distribution of tool rounds, not the mean:
| arm | 0-3 | 4-7 | 8-11 | 12-15 | 16+ | hit 80-step limit |
|---|---:|---:|---:|---:|---:|---:|
| 0.0.3 | 417 | 559 | 220 | **311** | 14 | 43 (2.7%) |
| PR #6 | 412 | 597 | 174 | 9 | 2 | 370 (23.7%) |
| thresholds reverted | 389 | 609 | 175 | **7** | 1 | 383 (24.5%) |
At the same threshold of 12 the pile past the threshold did not come back. In 0.0.3, 325 of 1521 recorded attempts (21.4%) reach 12 rounds and 145 of them succeed — 12.4% of all successes of that arm live in the band where the hard order has already fired. In PR #6 that band holds 11 attempts and 7 successes.
## Suggested fix
Keep `_nudge_count` for the grep/error-loop nudges (capping them at 2 is reasonable), but let the **budget** nudge re-inject while its condition holds — that is the safety valve. Whatever the shape, it needs its own measurement: the combination "PR #6 gate + repeatable nudge" has never been run.
## Workaround until then
Pin `deepagents-gigachat==0.0.3` when running the profile against a weaker model.
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Évaluation
Cette issue n'a pas encore été évaluée.