anthropics / anthropics/claude-code
[BUG] Prompt cache collapses to the system+tools floor on every resumed turn once the conversation exceeds ~80k tokens and >1 min has passed — 2.1.228+, Bedrock, Agent SDK headless
- Langage dominant
- Python
- Étoiles
- 145k
- Forks
- 23.1k
- Métriques de merge des PR
- Métriques de PR en attente
Description
## Preflight Checklist
- [x] I have searched existing issues and this hasn't been reported yet (see Related issues below — the closest ones differ in a way that matters)
- [x] This is a single bug report
- [x] I am using the latest version of Claude Code
---
## What's Wrong?
On a multi-turn headless workload (Claude Agent SDK → bundled CLI → Amazon Bedrock, `--resume` per turn), the first API call of a resumed turn stops reading the conversation from cache and re-writes it in full. Only the shared `system` + tool-definition prefix is served from cache.
It reproduces on **two conditions ANDed**, measured over 1,100+ resume boundaries of real production traffic:
**Conversation size** (`cache_read + cache_creation` of the previous turn's last call)
| conversation size | resumes that fell to the floor |
|---|---|
| < 80k | **0 / 443 (0.0%)** |
| 80–120k | 95 / 209 (45.5%) |
| 120–160k | 120 / 243 (49.4%) |
| 160k+ | 181 / 461 (39.3%) |
**Idle gap** (time between the previous turn's last API call and the resumed turn's first call)
| gap | fell to the floor |
|---|---|
| 0–1 min | **5 / 465 (1.1%)** |
| 1–5 min | 140 / 427 (32.8%) |
| 5–15 min | 97 / 219 (44.3%) |
| 15–60 min | 42 / 121 (34.7%) |
| > 60 min | **112 / 124 (90.3%)** |
Under 80k it never happens, at any gap. Back-to-back turns (<1 min) almost never trigger it, at any size. Both conditions together are the ordinary shape of a Slack-style assistant: a few tool-using turns push past 80k, and humans take minutes between questions.
**This is not a TTL expiry.** `cache_creation` is 100% `ephemeral_1h_input_tokens` on every record (we set `ENABLE_PROMPT_CACHING_1H=1`), so the client *is* asking for the 1-hour TTL and getting it. And Bedrock honors it — see the control experiment below. What changes is the request: on a fallen turn the rebuilt request is **smaller than the version that was cached** — `cache_creation − (expected delta)` has a median of **−6.0k** tokens (p25 −17.7k). The prefix no longer matches because content is missing from it, not because the cache went away.
There is a second-order effect: when call 1 of a turn falls to the floor and re-writes the history (mean `cache_creation` ≈ 98k), **call 2 of the same turn frequently re-writes the same history again** (p50 96.4k). We see 71 such double payments; all of them are call 2, never call 3 or later.
**Cost impact:** this is the dominant term in a +63% month-over-month bill on the affected workload (~$500/month). Per-turn cost went from $0.644 to $1.545 while the number of turns *fell* 10%.
## What Should Happen?
A resumed turn should send a request whose prefix is byte-identical to what the previous turn cached, so `cache_read` continues from the previous conversation length and only the new exchange is written. Conversation size and idle time should not change what the client serializes.
Concretely: at 120k conversation and a 10-minute gap, the resumed turn should read ~120k and write a few thousand tokens. Today it reads 53k (system+tools only) and writes ~67k.
## Error Messages/Logs
No errors. The only signal is `message.usage` in the session transcript.
Healthy resume (CLI 2.1.252, conversation 92k, gap ~30s — reproduced on a canary runtime):
```
turn N last call: cache_creation= 1,462 cache_read= 92,144
turn N+1 first call: cache_creation= 1,302 cache_read= 92,144 ← continues
```
Fallen resume (CLI 2.1.237, production, conversation 121k, gap ~8 min):
```
turn N last call: cache_creation= 4,345 cache_read=112,217
turn N+1 first call: cache_creation= 60,635 cache_read= 53,896 ← system+tools only
turn N+2 first call: cache_creation= 65,755 cache_read= 53,896 ← same floor again
```
`53,896` is exactly this deployment's `system` + tool-definition prefix. Across the fallen turns the `cache_read` value clusters tightly at 53,305–54,058 — it is always that floor, never a partial match.
### Control experiment: Bedrock honors the 1h TTL
To rule out the provider, we sent a byte-identical 117k prefix with `cache_control: {type: "ephemeral", ttl: "1h"}` directly via `bedrock-runtime:InvokeModel` (`global.anthropic.claude-sonnet-5`, ap-northeast-1), no Claude Code involved:
```
t=0s cache_creation=117,017 cache_read= 0
t=20s cache_creation= 0 cache_read=117,017
t=90s cache_creation= 0 cache_read=117,017
t=5m cache_creation= 0 cache_read=117,017
t=10m cache_creation= 0 cache_read=117,017
```
Same region, same account, same model, a size well past the 80k threshold, gaps well past 1 minute — **full hit every time**. The provider keeps a 1h-TTL entry exactly as asked. The regression is on the client side.
### Version boundary
Same day, same workload, three CLI versions running concurrently (rolling deploys leave older containers alive for a while):
```
2026-08-12 2.1.207 0 / 29 = 0.0%
2026-08-12 2.1.226 0 / 15 = 0.0%
2026-08-12 2.1.228 7 / 27 = 25.9%
```
By generation, over all resumes with a matching version:
```
2.1.207 – 2.1.227 0 – 4%
2.1.228 25.9 – 33.3%
2.1.229 21.8 – 25.9%
2.1.232 37.0%
2.1.233 – 2.1.238 20.0 – 31.1%
2.1.239 – 2.1.252 25.3 – 40.0% (still present on the latest)
```
`2.1.248` shipped *"Fixed the `ScheduleWakeup` tool definition changing between a session and its `--resume`, causing a full prompt-cache miss on the resumed session's first turn"* — same shape, but that fix did not resolve this (it is scoped to accounts in usage overage; this deployment is Bedrock).
### Ruled out
Each of these was measured, not assumed:
- **Session file transport / container reuse** — this deployment resumes by downloading the transcript from S3 when the container is new, or reusing the local file when it is not. Floor rate is **44.0%** (S3 download, n=432) vs **43.7%** (local file, n=158). Not a transport problem.
- **Concurrency** — no two sessions ever share a container within ±5 min (0 of 592). Not a race.
- **Compaction / microcompact** — 0 occurrences in 60 sampled sessions; no `isCompactSummary`, no sidechains.
- **Cache miss / TTL expiry** — `cache_read == 0` stays at 1–2% across all versions; every fallen turn still reads the 53k floor, so a cache entry existed.
- **`totalTokensReminder`** (#90018) — the `` attachment only starts appearing in our transcripts on 8/17, but the floor rate is already 23–34% on 8/13–8/16. Splitting the C period on that date gives 23.8% (no reminder) vs 26.1% (reminder). A local A/B with `totalTokensReminder: "off"` changed nothing.
- **Our own application changes** — the deploy that carried 2.1.228 also carried unrelated application code; stratifying by conversation size makes that effect vanish while the version effect survives.
- **Model / pricing** — happens on both Opus and Sonnet tiers, at the same rate.
## Steps to Reproduce
1. Run the Claude Agent SDK (Python) headless against Amazon Bedrock, one process per turn, resuming with `resume=` (`ClaudeAgentOptions.resume`). Set `ENABLE_PROMPT_CACHING_1H=1`.
2. Have a tool-using conversation until the request size passes **~80k tokens** (a handful of turns with real tool results).
3. Wait **more than 1 minute** — a few minutes makes it near-certain, an hour makes it ~90%.
4. Send a follow-up turn on the same session id.
5. Read `message.usage` of the resumed turn's first API call.
**Expected:** `cache_read` ≈ the previous turn's `cache_read + cache_creation`; `cache_creation` in the low thousands.
**Actual:** `cache_read` = the `system` + tool-definitions length only; `cache_creation` = the rest of the conversation.
Notes on reproducing:
- Back-to-back turns do **not** reproduce it. We failed to reproduce for a while precisely because our test harness ran follow-ups with no delay (0/4 and 0/10 at 92k with sub-minute gaps, on the same CLI build that shows 30–44% in production).
- Staying under 80k does **not** reproduce it either.
- No subagents, no Agent Teams, no forks, no images, no compaction are involved. This is a plain multi-turn conversation.
## Claude Model
Other (Bedrock — `claude-opus-5` and `claude-sonnet-5` via Application Inference Profiles; both affected)
## Is this a regression?
Yes, this worked in a previous version
## Last Working Version
2.1.227
## Claude Code Version
2.1.252 (Claude Code) — bundled by `claude-agent-sdk` 0.2.149. First bad: 2.1.228 (bundled by SDK 0.2.136).
## Environment
- Platform: Linux container (Amazon Bedrock AgentCore Runtime), one container per turn or reused
- Provider: Amazon Bedrock, ap-northeast-1, Application Inference Profiles
- Entry point: `claude-agent-sdk` (Python) headless, `ClaudeAgentOptions(resume=..., setting_sources=["project"], skills=[...])`
- 36 in-process SDK MCP tools, `ENABLE_TOOL_SEARCH=false` (tool definitions ride the cached prefix)
- `ENABLE_PROMPT_CACHING_1H=1`
- Session transcripts synced to S3 between turns (ruled out above)
## Related issues
- #84253 — 2.1.218+ stopped requesting the 1h TTL. **Different**: we *do* get the 1h TTL (100% `ephemeral_1h_input_tokens`) and still fall.
- #87215 — parked subagent re-caches its whole context, only the system prefix survives, within TTL. Same shape, but subagent-specific; ours is the main conversation.
- #80604 — Agent Teams delivery triggers a full rewrite, "only system+tools survive".
- #87966 — `cache_read` pinned to the stable-prefix boundary, 89 full-context rewrites over 9 days.
- #90018 — `totalTokensReminder` cache floor in tool loops (tested and excluded above).
- #86244 — a background auto-update invalidates existing sessions' cache. Explains a one-time rewrite at a version change, not a steady 30–44%.
## Additional Information
Happy to provide per-request `(cache_creation, cache_read, timestamp, version)` series for the fallen turns, or run a specific instrumented build. We cannot share conversation content, but every number above is derived from `message.usage` and timestamps only.
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Piste de recherche
Start with the Claude Agent SDK Python entry point using ClaudeAgentOptions(resume=...) and reproduce the issue with a conversation above 80k tokens and a gap over one minute. Compare message.usage across resumed turns and versions 2.1.227 and 2.1.228+; done means the resumed request preserves the prior cached prefix and reports high cache_read with only the new exchange in cache_creation.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- aws, python
- Domaine
- cli, cloud
- Type d'issue
- Bug
- Difficulté
- 5/5
- Temps estimé
- Plus d'une semaine
- Activité
- Active
- Clarté
- Plutôt claire
- Accessibilité débutants
- 38/100