anthropics / anthropics/claude-code

[BUG] Workflow keeps spawning agents after a terminal "session limit" error, and resumeFromRunId re-runs them into a second exhaustion — both report status: completed

Aperta
#94,012 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
area:agents area:cost bug has repro platform:macos
Lingua principale
Python
Stelle
145k
Fork
23.1k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

### What's Wrong?

A Workflow run keeps spawning agents after the session quota is already gone, and `resumeFromRunId` re-runs the dead ones into a second exhaustion. Neither run aborts, and both report `status: completed`.

Once an agent dies with `You've hit your session limit · resets

Measured across two runs of one workflow (same `runId`, second launched with `resumeFromRunId`):

| | agents | subagent tokens | agents killed by the quota error | wall clock | reported status |
|---|---|---|---|---|---|
| run 1 | 108 | 8,527,469 | 34 | 51 min | `completed` |
| run 2 (resume) | 108 | 8,052,689 | 42 | 56 min | `completed` |

From the shared `journal.jsonl` (424 records): after the **first** agent failure, **133 more agents were started, 75 of which failed.**

Three separable defects:

1. **No circuit breaker on a terminal quota error.** A quota stop is not a transient per-agent failure, but it is handled as one. Everything queued behind it is spawned and killed.
2. **Resume amplifies instead of protecting.** The resume correctly replayed the cached research agents, then re-ran every failed verifier — into a *different* reset window (`resets 6pm` → `resets 12am`), exhausting that one too. Net effect of retrying: the spend doubled and the result was still partial. Adjacent to #91449, but that asks for checkpointing *inside* an agent; this is about the wasted re-spawn.
3. **`status: completed` on a run that lost 39% of its agents.** The failure list is in a `` block, but the headline says completed. A model reading that summary treats a partial result set as the finished answer — in my case 8 of 52 verifier claims came back with zero votes and were silently indistinguishable from verified ones until I inspected the JSON by hand.

Cost in context: the workflow was researching one factual API question that four greps and an SDK header answered. 16.6M subagent tokens, two full session windows, and the user's Fable quota for the day. The sizing half of that is #92631 and #91942; this report is about the part where the harness had already been told, 34 times, that it was out of budget and carried on.

### What Should Happen?

- Treat a usage/session-limit error from any agent as **run-terminal**: cancel the queue, stop spawning, return partial results immediately with `status: aborted` (or similar) and a reason.
- Have `resumeFromRunId` refuse to start, or warn, when the quota window that killed the prior run has not reset.
- Distinguish `completed` from `completed with N terminal failures` in the task notification headline, not only in the `` block, so the calling model does not read a partial result set as complete.

### Error Messages/Logs

Repeated 34× in run 1 and 42× in run 2, one per killed agent, identical apart from the label:

```
[refute:action-button] failed: You've hit your session limit · resets 6pm (America/Vancouver)
[refute:capture-event] failed: You've hit your session limit · resets 6pm (America/Vancouver)
...
[refute:locked-camera-capture] failed: You've hit your session limit · resets 12am (America/Vancouver)
```

Journal records for failed agents carry only `{type, key, agentId}` — no error text — so the reason a given agent died is not recoverable from `journal.jsonl`, only from the task notification.

### Steps to Reproduce

1. Session with ultracode on, so a workflow is authored by default.
2. Run a `pipeline()` whose second stage fans out several verifier agents per item from stage one (mine: 4 research agents → 2 refuters per claim → 108 agents total).
3. Let it run until the session/usage limit is reached mid-run.
4. Observe: agents continue to be spawned and immediately fail with the same limit error until the script's agent list is exhausted; the run finishes with `status: completed`.
5. Relaunch with `{scriptPath, resumeFromRunId}`. Cached agents replay, the previously-failed ones re-run, and the same exhaustion repeats in the next window.

### Claude Model

Fable 5.1 (`claude-fable-5-1`) as the session model; workflow agents inherited it.

### Is this a regression?

Not that I can tell — I have not run a workflow of this size into a quota wall before.

### Claude Code Version

2.1.265

### Platform

Claude Desktop app (Code tab)

### Operating System

macOS (Darwin 25.5.0)

### Terminal/Shell

zsh

### Additional Information

Related but distinct: #92631 (no effective agent ceiling under ultracode) and #91942 (ultracode exhausting the session limit) both cover *how many* agents get spawned. #91449 covers resuming an interrupted agent's own work. None of them cover the orchestrator continuing to spawn into a known-exhausted quota, or the `completed` status on a run that lost a third of its agents to it.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Start by finding the workflow orchestration path for pipeline() runs, resumeFromRunId handling, agent failure handling, journal.jsonl records, and the task notification status. Reproduce with a fan-out workflow that hits the session-limit error. Done means a terminal quota error stops further spawns, resume warns or refuses before reset, and the reported status distinguishes partial/aborted runs from completed ones.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
ai-infra-agents, cli, macos, zsh
Ambito
ai-infra-agents, cli
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Attiva
Chiarezza
Abbastanza chiara
Idoneità per principianti
48/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.