anthropics / anthropics/claude-code

[BUG] Resuming a background subagent misses its prompt cache: messages_changed, no thinking blocks

Open
#94,728 0 comments 1 reaction 0 assignees View on GitHub
area:agents area:cost bug has repro platform:linux platform:wsl
Dominant language
Python
Stars
145k
Forks
23.1k
PR merge metrics
PR metrics pending

Description

### Preflight Checklist

- [x] I have searched [existing issues](https://github.com/anthropics/claude-code/issues?q=is%3Aissue%20state%3Aopen%20label%3Abug) and this hasn't been reported yet
- [x] This is a single bug report (please file separate reports for different bugs)
- [x] I am using the latest version of Claude Code

### What's Wrong?

The first message sent to a background subagent after it has ended its turn produces a request the API reports as a `messages_changed` cache miss, and the subagent re-writes almost its whole context at cache-write prices. The 1-hour cache lifetime was in force (`ENABLE_PROMPT_CACHING_1H=1`; the transcripts show `ephemeral_1h` writes) and both resumes fell inside it.

Two resumes measured on 2.1.273, both custom agent definitions run with the `Agent` tool and `run_in_background`, model `claude-opus-5`, resumed by `SendMessage` after the completion notification. Figures from `message.usage` on the subagent transcripts, assistant records deduplicated by `message.id`:

| agent | last live request (total input) | idle | resume request (total input) | `cache_read_input_tokens` | `cache_creation_input_tokens` | shrink | thinking tokens produced before the stop | API diagnostics |
|---|---|---|---|---|---|---|---|---|
| `a1d90c482282ecd58` | 312,205, `end_turn` | 54.5 min | 259,599 | 16,381 | 243,214 | 52,606 | 53,084 | `messages_changed`, `cache_missed_input_tokens` 199,150 |
| `a0ed53adb38b3758c` | 462,177, `end_turn` | 49.4 min | 415,879 | 17,253 | 398,622 | 46,298 | 56,664 | `messages_changed`, `cache_missed_input_tokens` 323,464 |

Three things pin the miss to the message history and, on this version, to the thinking blocks:

1. **The diagnostics.** `diagnostics.cache_miss_reason.type` is `messages_changed`, which the cache diagnostics page defines as "The model, system, and tools all match, but an earlier entry in `messages` was altered, reordered, or removed rather than appended to." The read on each resume (16,381 and 17,253) equals what a fresh agent of the same definition reads for its tools-plus-system layer, so the whole system layer matched and the divergence is at the start of `messages`.
2. **The size arithmetic.** The resumed request is smaller than the last live request by the agent's summed thinking tokens. First agent: 312,205 minus 53,084 thinking, plus about 585 tokens of resume message and 74 of final sentence, predicts 259,780 against 259,599 observed (0.07% off). Second agent: the same arithmetic predicts 406,538 against 415,879 observed, so about 9,300 tokens more were sent than "history minus thinking" accounts for; that residual is open. During the live run the thinking was in the prompt and read from cache (one step produced 13,828 output tokens, 13,475 of them thinking, and the next request grew by 15,545).
3. **The API dropped nothing.** The resume response carries `input_transformations: []` under the `thinking-binding-controls` beta and `context_management: null`, while the request the harness sends carries `context_management.edits = [{"type": "clear_thinking_20251015", "keep": "all"}]` on every request. So the thinking blocks were not in the resume request.

The cheap paths for comparison, same agents and version: four messages delivered while the agents were still working arrived as `queued_command` attachments inside the open tool-use turn and cost 2,013 to 4,373 tokens of writes with reads intact; the second and later messages after each resume were cheap again (writes of about 14K then 1.5K). Only the first message after `end_turn` pays.

### What Should Happen?

A follow-up to a stopped background subagent inside the cache lifetime should be a cache read of the history the original run warmed, with its thinking blocks intact. Instead the resumed request reads only the agent's own tools-plus-system layer (16,381 and 17,253 tokens, what a fresh agent of that definition reads) and re-writes the rest (243,214 and 398,622), at 54 and 49 minutes idle under `ttl 1h`, and on 2.1.273 the rebuilt history is smaller by the agent's thinking.

The sub-agents page says: "Resumed subagents retain their full conversation history, including all previous tool calls, results, and reasoning." and "The resumed run keeps the tool set from where the subagent first ran and can keep reading the prompt cache the original run warmed." The prompt caching page says: "**Resumed subagents**: when Claude resumes a subagent, the resumed run's first request can read the cache the original run warmed."

Expected, then: the first request after a resume inside the lifetime reads the cache the original run built, and the resumed history carries the agent's thinking blocks. Observed: it reads only the tools-plus-system layer, re-writes the rest, and on 2.1.273 the rebuilt history is smaller by the agent's thinking.

### Error Messages/Logs

```shell
No error is emitted. The API's verdict is on the first assistant record after the resume in the subagent transcript:

message.usage.cache_read_input_tokens 16381
message.usage.cache_creation_input_tokens 243214
message.diagnostics.cache_miss_reason {"type": "messages_changed", "cache_missed_input_tokens": 199150}
message.input_transformations []
message.context_management null
```

### Steps to Reproduce

1. Put `ENABLE_PROMPT_CACHING_1H=1` in the `env` block of `settings.json` (or set `subagentPromptCacheTtl` to `1h`) so the lifetime is not the variable.
2. From an interactive session, launch a background subagent with a custom definition (not Explore or Plan, which return no agent ID) on a task that makes it think and build context: read several files, reason about them, run a few tool calls, report back. The two measured agents ran on `claude-opus-5` and had 53K and 57K thinking tokens when they stopped.
3. Wait for the completion notification. Do not send the follow-up on the hand-back message: a message sent before the agent's final `end_turn` is answered "Message queued for delivery to at its next tool round", and if the agent has no further tool round it is never delivered and no resume happens.
4. Within the hour, `SendMessage` one short follow-up to the agent ID.
5. Open the subagent transcript at `~/.claude/projects///subagents/agent-.jsonl`. Deduplicate assistant records by `message.id` (streamed blocks repeat the same input figures). Find the `user` record with `origin.kind == "coordinator"` (its text begins "The coordinator sent a message while you were working:") whose preceding assistant record has `stop_reason: "end_turn"`. A message that arrived while the agent was working is an `attachment` record of type `queued_command` and is not this case.
6. On the next assistant record read `message.usage.cache_read_input_tokens`, `message.usage.cache_creation_input_tokens`, `message.diagnostics.cache_miss_reason` and `message.input_transformations`. Sum `output_tokens_details.thinking_tokens` over every assistant record before the resume and compare with (last live request's total input) minus (resume request's total input).
7. To see the request itself, run the parent with `OTEL_LOG_RAW_API_BODIES=file:` and diff the last pre-stop `*.request.json` against the resume one, ignoring `cache_control` markers. The bodies contain the whole conversation, so keep them local.

What to expect if the bug is present: the read collapses to the agent type's own tools-plus-system layer (12K to 25K here), the write is roughly the previous request's total input, the diagnostic is `messages_changed`, `input_transformations` is `[]`, and on 2.1.273 the shrink equals the thinking sum within the size of the resume message. On 2.1.269 the same miss occurred with no shrink (see Additional Information). A subagent that produced no thinking blocks resumed with a full hit here, so the task in step 2 has to produce thinking.

### Claude Model

Opus

### Is this a regression?

I don't know

### Last Working Version

Unknown. Not seen hitting on 2.1.269 or 2.1.273 for an agent with thinking blocks.

### Claude Code Version

2.1.273 (Claude Code)

### Platform

Anthropic API

### Operating System

Ubuntu/Debian Linux

### Terminal/Shell

WSL (Windows Subsystem for Linux)

### Additional Information

### Claude Model (Specifics)

Other: `claude-opus-5` for the two measured resumes. The control that hit was `claude-sonnet-5` with no thinking blocks.

### Is this a regression? (Expanded)

I don't know. The same first-message miss occurred on 2.1.269 without the size change, so the shrink is version-specific and the miss is not.

**Control that hit.** A `general-purpose` background subagent on `claude-sonnet-5` ran `ls`, handed back, and 8 seconds after its `end_turn` received one follow-up. It produced no thinking blocks. The resume request hit: read 37,971 (the whole of the previous request, 37,794 read plus 177 written), wrote 125, `diagnostics: null`. With `OTEL_LOG_RAW_API_BODIES` set, a diff of the last pre-stop body against the resume body showed `tools` identical, `system` identical apart from a per-request billing line in an uncached preamble block, messages 0 to 6 byte-identical, and message 7 (the trailing `` reminder sent as a `role: system` message) changed from a one-element text-block array with a `cache_control` marker to a plain string, which the API still matched. So a plain resume with no thinking and no mid-run messages does not miss, and that reminder's re-serialisation is harmless. A body capture straddling a thinking-bearing resume has not been done; the identification above rests on the size arithmetic and `input_transformations: []`.

**The same miss on 2.1.269 without the shrink.** On 2.1.269, two days earlier, every first follow-up after an `end_turn` that missed collapsed its read to 24,784 tokens with `messages_changed`, and every second and later follow-up hit (writes of 1K to 5K, `diagnostics: null`). There the resumed request did not shrink by the thinking sum: size changes were between -2K and +23K against 16K to 66K of thinking produced. Of 20 first follow-ups inside the lifetime, 15 re-paid, 2 hit cleanly and 3 partially, with misses at 0.3 minutes idle and hits at 39.9 minutes idle, so idle time is not the variable. Two alterations are in play across the two versions, or one alteration with a changed size signature. `cache_missed_input_tokens` sat at 0.81 to 0.84 of the message tokens in every case, including one that hit the cache almost entirely (read 235,537 of 237,826, still `messages_changed` with 176,002 "missed"), so it does not locate the divergence.

**Where the resume is built.** The 2.1.273 binary's background resume path (`resumeAgentBackground`) reloads the agent's messages from its disk transcript, deletes each attachment's stored `rendered` text and re-renders it, and passes the history through its normalisers before the request is built. The transcripts carry no `thinking_stripped` marker.

**Changelog entries on sibling cases**, quoted from the current CHANGELOG.md:

- 2.1.265: "Fixed resuming a foreground-spawned subagent changing its tool list and system prompt prefix, which broke prompt-cache reuse for that agent" and "Fixed agent teammates and resumed subagents moving SubagentStart hook context and preloaded skills out of the prompt prefix on later turns, which broke prompt-cache reuse".
- 2.1.267: "Fixed a tool that disappears mid-conversation, from a disconnected MCP server or an upgrade, rewriting the tool list and discarding earlier thinking"; "Fixed prompt-cache misses and dropped extended thinking when a claude.ai connector's tools change between a session and its resume"; "Fixed resumed sessions rewriting earlier MCP tool announcements (and dropping extended thinking) before their connectors reconnect".
- 2.1.269: "Fixed the prompt cache being partially invalidated on the turn after a response was cut off at the output-token limit and automatically resumed" and "Fixed a case where resuming a session after interrupting Claude mid-thought could change how earlier context was re-sent, hurting prompt-cache reuse".
- 2.1.273: "Fixed `/login`, `/upgrade`, and `/extra-usage` discarding earlier thinking from the conversation, which forced a full prompt-cache rewrite on the next request".
- 2.1.260: "Added a likely cause for prompt-cache misses (e.g. tool definitions or system prompt changed, idle past the TTL) to `/cost` and the status line's `prompt_cache` field".

The background-spawned, definition-based subagent resume has the same shape as the case the 2.1.265 entry fixed for foreground subagents (a resume whose first request no longer matches the cached prefix) and still misses on 2.1.273, with the divergence in `messages` rather than tools or system.

**Related, not duplicate:**

- #88444: forks (`subagent_type: "fork"`), a deterministic `messages_changed` miss after any tool loop with the read pinned to the system-plus-tools floor; its author says what the client serialises differently is unidentified. Ours is a definition-based background agent, not a fork, with the 1-hour lifetime in force (the transcripts show `ephemeral_1h` writes), and on 2.1.273 the difference is measured: the thinking blocks.
- #88412 (closed into #88444): the idle-fork wake variant, `cache_read` pinned to a fixed boundary. Ours is not a fork and the read is the agent's own definition layer.
- #76606: an old hook reminder changing shape mid-session in a long main conversation, no resume involved. Ours is the resume path of a stopped background subagent, and the one reminder re-serialisation we captured (array to plain string) did not miss.
- #44724 (closed): the first `SendMessage` resume of a fresh subagent reading `cache_read` 0 because `system[1]` swapped identity strings. Ours reads the full system layer; the divergence is in `messages`.
- #93490: `--resume` of a main session replaying the session-start context message as a plain string. Ours is a subagent, and the plain-string re-serialisation we captured did not miss.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the background resumeAgentBackground path and compare the stored subagent transcript under ~/.claude/projects///subagents/agent-.jsonl with the captured request JSON from OTEL_LOG_RAW_API_BODIES. Check the cache diagnostics, message history, input transformations, and thinking blocks; done means a follow-up within the 1-hour lifetime reuses the warmed history and preserves thinking blocks without a messages_changed miss.

Written by the indexing model from the issue text.

Assessment

Tech stack
json, linux, python, shell
Domain
backend-api-design, cli, devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.