Workflow resumeFromRunId re-runs every completed agent after the first parallel fan-out (40 redundant opus agents, 231M tokens)

**Claude Code 2.1.274, Workflow tool, session model Claude Fable 5.1, subagents on opus / sonnet / haiku. WSL2 Ubuntu on Windows 11. 2026-09-17, 05:00–07:25 UTC (2026-09-16, 22:00–00:25 PDT).**

## Summary

`Workflow({scriptPath, resumeFromRunId})` is documented as replaying completed `agent()` calls from cache: *"the longest unchanged prefix of agent() calls returns cached results instantly; the first edited/new call and everything after it runs live."* In a script that uses `parallel()` or `pipeline()`, that prefix is broken on every resume, because concurrent `agent()` calls start in a different order each run. The cache therefore misses from the first fan-out onward and every downstream agent runs again at full cost, even though the script and the arguments feeding those prompts did not change.

The same tool description also says *"Completed agent() calls with unchanged (prompt, opts) return their cached results instantly"* and *"Same script + same args → 100% cache hit."* Those two sentences describe content-keyed caching. The behaviour observed is position-keyed. For any real workflow (which is to say, any workflow that fans out), the first two sentences are false.

Result on one build run: **40 redundant agent runs, 230.8M tokens (217.8M cache-read, 12.3M cache-write, 0.66M output), 28 % of the run's raw tokens and about 29 % of its weighted cost**, spent re-reading and re-verifying files that were already written and committed. The redundant agents were all on opus. The user noticed only because the agents' narration said "already written, just verifying."

## Timeline

| UTC (PDT) | Event |
|---|---|
| 2026-09-17 01:00 (16 Sep 18:00) | Run launched with `args.stopAt = 'G1'`. Phase 1: 5 agents in one `parallel()`. All complete. Script returns at gate G1. |
| 01:15 (18:15) | `resumeFromRunId` with `stopAt = 'G2'`. The 5 phase-1 agents replay from cache (correct). Phases 2–4 launch: a scaffold agent, then two `parallel()` fan-outs of 15 + 11 doc agents. |
| 01:48 (18:48) | Session usage guard trips; run stopped with `TaskStop`. 25 of 33 agents done, 8 in flight (expected loss). |
| 05:02 (22:02) | `resumeFromRunId` again, same `stopAt = 'G2'`, script unchanged. **Expected:** 25 cached, 8 live. **Observed:** the 15 `houdini-dev` doc agents replay, but the 4 completed `houdini-pdg-dev` doc agents run again (12 redundant starts in the 05:00 hour, including the scripts/close-out agents that followed). The fan-out order had changed. |
| 06:50 (23:50) | Script returns at G2: 53 agents "done". (Only 33 had been launched before, 8 lost: the tally already contained 12 extra runs. Not noticed.) |
| 07:00 (00:00) | One prompt edited in a late phase (the audit-fix agent, phase 6, not yet run). `resumeFromRunId` with `stopAt = 'G3'`. **Expected:** phases 1–4 cached, phase 5 onward live. **Observed:** every agent from the phase-3 PDG fan-out onward starts again — 24 redundant starts in the 07:00 hour: 11 PDG docs (third run), 14 GUI docs (second run), both scripts agents, both close-outs. Each re-read the committed docs, "verified" them, and returned. |
| 07:25 (00:25) | User asks why agents say "already written, I'm just verifying." Run stopped. |

Journal evidence (`journal.jsonl`, same run id throughout): 53 distinct labels, 93 `started` events, 0 duplicate keys — every re-run got a **new cache key** for an identical prompt and identical opts. Redundant `started` events by hour: 05 UTC 12, 06 UTC 4, 07 UTC 24.

## Token cost (summed from the per-agent transcripts' `usage` blocks)

| | Agents | Cache read | Cache write | Output | Raw total |
|---|---|---|---|---|---|
| Needed (first run of each label) | 53 | 564.5M | 23.6M | 2.48M | 590.5M |
| **Redundant (second/third run)** | **40** | **217.8M** | **12.3M** | **0.66M** | **230.8M** |

Weighted at list prices (cache read 0.1×, cache write 1.25×, output 5×): about 40M input-equivalent tokens wasted against about 98M needed. The account's 5-hour window read 250M weighted at the stop; roughly 70–100M of that was the redundant work.

## Why the cache missed

The cache key is derived from call *position* in the launch sequence, not from `(prompt, opts)` content. `parallel(thunks)` and `pipeline(items, …)` start their agents concurrently, so the order in which `agent()` calls are recorded depends on scheduling. Two runs of the same script with the same args produce different launch orders inside every fan-out; the "longest unchanged prefix" ends at the first call whose position holds a different prompt than last time. From there on nothing is cached.

An edit anywhere in the script (even to a prompt that has never executed) has the same effect on everything after the first fan-out — but note that the 05:02 resume had **no** script edit and still re-ran completed agents.

## Why this is severe

- **It defeats the feature's stated purpose.** Resume exists for long runs that hit session limits or need gates. Those are exactly the runs with fan-outs. The documented pattern (return at a gate, resume with `resumeFromRunId`) costs more than relaunching would, because the "verify" pass on an already-written artifact is a full read of it.
- **It is silent.** The tool result on resume says only "completed agents return cached results". `/workflows` shows agents running with the same labels as before; nothing marks them as re-runs. The only signals are the journal (duplicate labels with fresh keys) and the agents' own narration.
- **The session model did not catch it either.** Fable 5.1 read the resume contract, took the content-keyed sentences at face value, designed the gate/resume pattern on them, and after two resumes did not compare the completion tally (53) against launched-so-far (33) plus known losses (8). The doc string invited that reading; the model should still have verified it. Both halves of the product failed the user here.

## Reproduction

```javascript
export const meta = { name: 'resume-cache-repro', description: 'show parallel fan-out breaking resume cache', phases: [{ title: 'A' }, { title: 'B' }] }
phase('A')
const a = await parallel([1, 2, 3, 4, 5, 6].map((i) => () =>
agent(`Return the single word ok-${i}.`, { label: `a${i}`, model: 'haiku' })))
if (args.stopAt === 'A') return { a }
phase('B')
const b = await agent('Return the single word done.', { label: 'b', model: 'haiku' })
return { a, b }
```

1. `Workflow({script, args: {stopAt: 'A'}})` — 6 agents run.
2. `Workflow({scriptPath, resumeFromRunId, args: {stopAt: 'B'}})`.
3. Expected: `a1`–`a6` cached, only `b` runs. Observed (in our run, with the same shape at scale): some or all of `a1`–`a6` run again; `journal.jsonl` shows their labels `started` twice with different `key` values. Repeat step 2 a few times to see the count vary with scheduling.

## What would fix it

1. **Key the cache by content**, `(prompt, opts, phase)`, and match by set membership rather than sequence prefix. Order-independent replay is what the doc already promises.
2. Failing that, **say so in the tool description**: "resume caching is prefix-based; any `parallel()`/`pipeline()` breaks the prefix; use an explicit phase-skip argument instead." Remove the "unchanged (prompt, opts)" and "100 % cache hit" sentences, which are not true for scripts with fan-outs.
3. **Surface re-runs.** On resume, report `cached: N, live: M, of which M_dup previously completed with identical prompt`. A label that completed before and is starting again should be visible in `/workflows`.
4. **Budget stop on resume.** A resume that is about to launch more live agents than the original leg had remaining should pause and ask.

## Workaround we now use

An `args.startAt` phase number; each phase is wrapped in `if (!skip(n))` and finished phases are skipped by code. We no longer pass `resumeFromRunId` for anything downstream of a fan-out. After any resume, we count `started` per label in `journal.jsonl` before trusting the tally.

## Environment

- Claude Code 2.1.274 (CLI, WSL2 Ubuntu 22.04 on Windows 11, repo on `/mnt/c`)
- Session model: Claude Fable 5.1; workflow agents: opus (docs, close-outs), sonnet, haiku
- 16 CPUs → concurrency cap 14; workflow size guideline raised via `/config`
- Run: 53 distinct agents across 7 phases, one script file, `args` unchanged between the 01:15 and 05:02 resumes

Filed 2026-09-17 (PDT). Journal and per-agent transcripts available on request.

Start by running the JavaScript reproduction with Workflow, parallel(), and resumeFromRunId, then inspect journal.jsonl for duplicate labels and cache keys. Compare repeated runs with unchanged prompts and opts; done means either order-independent cache replay is implemented and verified, or the resume behavior and tool description clearly document the prefix limitation.

Written by the indexing model from the issue text.