anthropics / anthropics/claude-code

[BUG] Prompt cache collapses to the system+tools floor on every resumed turn once the conversation exceeds ~80k tokens and >1 min has passed — 2.1.228+, Bedrock, Agent SDK headless

Aperta
#91,151 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
api:bedrock area:agent-sdk area:core area:cost bug has repro
Lingua principale
Python
Stelle
145k
Fork
23.1k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

## Preflight Checklist

- [x] I have searched existing issues and this hasn't been reported yet (see Related issues below — the closest ones differ in a way that matters)
- [x] This is a single bug report
- [x] I am using the latest version of Claude Code

---

## What's Wrong?

On a multi-turn headless workload (Claude Agent SDK → bundled CLI → Amazon Bedrock, `--resume` per turn), the first API call of a resumed turn stops reading the conversation from cache and re-writes it in full. Only the shared `system` + tool-definition prefix is served from cache.

It reproduces on **two conditions ANDed**, measured over 1,100+ resume boundaries of real production traffic:

**Conversation size** (`cache_read + cache_creation` of the previous turn's last call)

| conversation size | resumes that fell to the floor |
|---|---|
| < 80k | **0 / 443 (0.0%)** |
| 80–120k | 95 / 209 (45.5%) |
| 120–160k | 120 / 243 (49.4%) |
| 160k+ | 181 / 461 (39.3%) |

**Idle gap** (time between the previous turn's last API call and the resumed turn's first call)

| gap | fell to the floor |
|---|---|
| 0–1 min | **5 / 465 (1.1%)** |
| 1–5 min | 140 / 427 (32.8%) |
| 5–15 min | 97 / 219 (44.3%) |
| 15–60 min | 42 / 121 (34.7%) |
| > 60 min | **112 / 124 (90.3%)** |

Under 80k it never happens, at any gap. Back-to-back turns (<1 min) almost never trigger it, at any size. Both conditions together are the ordinary shape of a Slack-style assistant: a few tool-using turns push past 80k, and humans take minutes between questions.

**This is not a TTL expiry.** `cache_creation` is 100% `ephemeral_1h_input_tokens` on every record (we set `ENABLE_PROMPT_CACHING_1H=1`), so the client *is* asking for the 1-hour TTL and getting it. And Bedrock honors it — see the control experiment below. What changes is the request: on a fallen turn the rebuilt request is **smaller than the version that was cached** — `cache_creation − (expected delta)` has a median of **−6.0k** tokens (p25 −17.7k). The prefix no longer matches because content is missing from it, not because the cache went away.

There is a second-order effect: when call 1 of a turn falls to the floor and re-writes the history (mean `cache_creation` ≈ 98k), **call 2 of the same turn frequently re-writes the same history again** (p50 96.4k). We see 71 such double payments; all of them are call 2, never call 3 or later.

**Cost impact:** this is the dominant term in a +63% month-over-month bill on the affected workload (~$500/month). Per-turn cost went from $0.644 to $1.545 while the number of turns *fell* 10%.

## What Should Happen?

A resumed turn should send a request whose prefix is byte-identical to what the previous turn cached, so `cache_read` continues from the previous conversation length and only the new exchange is written. Conversation size and idle time should not change what the client serializes.

Concretely: at 120k conversation and a 10-minute gap, the resumed turn should read ~120k and write a few thousand tokens. Today it reads 53k (system+tools only) and writes ~67k.

## Error Messages/Logs

No errors. The only signal is `message.usage` in the session transcript.

Healthy resume (CLI 2.1.252, conversation 92k, gap ~30s — reproduced on a canary runtime):

```
turn N last call: cache_creation= 1,462 cache_read= 92,144
turn N+1 first call: cache_creation= 1,302 cache_read= 92,144 ← continues
```

Fallen resume (CLI 2.1.237, production, conversation 121k, gap ~8 min):

```
turn N last call: cache_creation= 4,345 cache_read=112,217
turn N+1 first call: cache_creation= 60,635 cache_read= 53,896 ← system+tools only
turn N+2 first call: cache_creation= 65,755 cache_read= 53,896 ← same floor again
```

`53,896` is exactly this deployment's `system` + tool-definition prefix. Across the fallen turns the `cache_read` value clusters tightly at 53,305–54,058 — it is always that floor, never a partial match.

### Control experiment: Bedrock honors the 1h TTL

To rule out the provider, we sent a byte-identical 117k prefix with `cache_control: {type: "ephemeral", ttl: "1h"}` directly via `bedrock-runtime:InvokeModel` (`global.anthropic.claude-sonnet-5`, ap-northeast-1), no Claude Code involved:

```
t=0s cache_creation=117,017 cache_read= 0
t=20s cache_creation= 0 cache_read=117,017
t=90s cache_creation= 0 cache_read=117,017
t=5m cache_creation= 0 cache_read=117,017
t=10m cache_creation= 0 cache_read=117,017
```

Same region, same account, same model, a size well past the 80k threshold, gaps well past 1 minute — **full hit every time**. The provider keeps a 1h-TTL entry exactly as asked. The regression is on the client side.

### Version boundary

Same day, same workload, three CLI versions running concurrently (rolling deploys leave older containers alive for a while):

```
2026-08-12 2.1.207 0 / 29 = 0.0%
2026-08-12 2.1.226 0 / 15 = 0.0%
2026-08-12 2.1.228 7 / 27 = 25.9%
```

By generation, over all resumes with a matching version:

```
2.1.207 – 2.1.227 0 – 4%
2.1.228 25.9 – 33.3%
2.1.229 21.8 – 25.9%
2.1.232 37.0%
2.1.233 – 2.1.238 20.0 – 31.1%
2.1.239 – 2.1.252 25.3 – 40.0% (still present on the latest)
```

`2.1.248` shipped *"Fixed the `ScheduleWakeup` tool definition changing between a session and its `--resume`, causing a full prompt-cache miss on the resumed session's first turn"* — same shape, but that fix did not resolve this (it is scoped to accounts in usage overage; this deployment is Bedrock).

### Ruled out

Each of these was measured, not assumed:

- **Session file transport / container reuse** — this deployment resumes by downloading the transcript from S3 when the container is new, or reusing the local file when it is not. Floor rate is **44.0%** (S3 download, n=432) vs **43.7%** (local file, n=158). Not a transport problem.
- **Concurrency** — no two sessions ever share a container within ±5 min (0 of 592). Not a race.
- **Compaction / microcompact** — 0 occurrences in 60 sampled sessions; no `isCompactSummary`, no sidechains.
- **Cache miss / TTL expiry** — `cache_read == 0` stays at 1–2% across all versions; every fallen turn still reads the 53k floor, so a cache entry existed.
- **`totalTokensReminder`** (#90018) — the `` attachment only starts appearing in our transcripts on 8/17, but the floor rate is already 23–34% on 8/13–8/16. Splitting the C period on that date gives 23.8% (no reminder) vs 26.1% (reminder). A local A/B with `totalTokensReminder: "off"` changed nothing.
- **Our own application changes** — the deploy that carried 2.1.228 also carried unrelated application code; stratifying by conversation size makes that effect vanish while the version effect survives.
- **Model / pricing** — happens on both Opus and Sonnet tiers, at the same rate.

## Steps to Reproduce

1. Run the Claude Agent SDK (Python) headless against Amazon Bedrock, one process per turn, resuming with `resume=` (`ClaudeAgentOptions.resume`). Set `ENABLE_PROMPT_CACHING_1H=1`.
2. Have a tool-using conversation until the request size passes **~80k tokens** (a handful of turns with real tool results).
3. Wait **more than 1 minute** — a few minutes makes it near-certain, an hour makes it ~90%.
4. Send a follow-up turn on the same session id.
5. Read `message.usage` of the resumed turn's first API call.

**Expected:** `cache_read` ≈ the previous turn's `cache_read + cache_creation`; `cache_creation` in the low thousands.
**Actual:** `cache_read` = the `system` + tool-definitions length only; `cache_creation` = the rest of the conversation.

Notes on reproducing:
- Back-to-back turns do **not** reproduce it. We failed to reproduce for a while precisely because our test harness ran follow-ups with no delay (0/4 and 0/10 at 92k with sub-minute gaps, on the same CLI build that shows 30–44% in production).
- Staying under 80k does **not** reproduce it either.
- No subagents, no Agent Teams, no forks, no images, no compaction are involved. This is a plain multi-turn conversation.

## Claude Model

Other (Bedrock — `claude-opus-5` and `claude-sonnet-5` via Application Inference Profiles; both affected)

## Is this a regression?

Yes, this worked in a previous version

## Last Working Version

2.1.227

## Claude Code Version

2.1.252 (Claude Code) — bundled by `claude-agent-sdk` 0.2.149. First bad: 2.1.228 (bundled by SDK 0.2.136).

## Environment

- Platform: Linux container (Amazon Bedrock AgentCore Runtime), one container per turn or reused
- Provider: Amazon Bedrock, ap-northeast-1, Application Inference Profiles
- Entry point: `claude-agent-sdk` (Python) headless, `ClaudeAgentOptions(resume=..., setting_sources=["project"], skills=[...])`
- 36 in-process SDK MCP tools, `ENABLE_TOOL_SEARCH=false` (tool definitions ride the cached prefix)
- `ENABLE_PROMPT_CACHING_1H=1`
- Session transcripts synced to S3 between turns (ruled out above)

## Related issues

- #84253 — 2.1.218+ stopped requesting the 1h TTL. **Different**: we *do* get the 1h TTL (100% `ephemeral_1h_input_tokens`) and still fall.
- #87215 — parked subagent re-caches its whole context, only the system prefix survives, within TTL. Same shape, but subagent-specific; ours is the main conversation.
- #80604 — Agent Teams delivery triggers a full rewrite, "only system+tools survive".
- #87966 — `cache_read` pinned to the stable-prefix boundary, 89 full-context rewrites over 9 days.
- #90018 — `totalTokensReminder` cache floor in tool loops (tested and excluded above).
- #86244 — a background auto-update invalidates existing sessions' cache. Explains a one-time rewrite at a version change, not a steady 30–44%.

## Additional Information

Happy to provide per-request `(cache_creation, cache_read, timestamp, version)` series for the fallen turns, or run a specific instrumented build. We cannot share conversation content, but every number above is derived from `message.usage` and timestamps only.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Start with the Claude Agent SDK Python entry point using ClaudeAgentOptions(resume=...) and reproduce the issue with a conversation above 80k tokens and a gap over one minute. Compare message.usage across resumed turns and versions 2.1.227 and 2.1.228+; done means the resumed request preserves the prior cached prefix and reports high cache_read with only the new exchange in cache_creation.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
aws, python
Ambito
cli, cloud
Tipo di issue
Bug
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Attiva
Chiarezza
Abbastanza chiara
Idoneità per principianti
38/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.