anthropics / anthropics/claude-agent-sdk-python
Multi-turn headless resume falls back to the system+tools cache floor past ~82k tokens (CLI 2.1.228+, Bedrock) — same fingerprint as #974 / #1120
- Lenguaje dominante
- Python
- Estrellas
- 8.1k
- Forks
- 1.3k
- Métricas de merge de PR
- Métricas de PR pendientes
Descripción
Multi-turn headless resume: the conversation cache falls back to the system+tools floor once the conversation passes ~82k tokens (bundled CLI 2.1.228+, Bedrock). Same fingerprint as #974 and #1120, both fixed and closed before this window.
## What happens
Headless, one process per turn, `ClaudeAgentOptions(resume=)`, transcript restored from S3 between turns (the "Move the session file" route from the sessions docs). On the first API call of a resumed turn, `cache_read_input_tokens` drops to exactly the `system` + tool-definitions length and the whole conversation is re-written.
```
turn N last call: cache_creation= 4,345 cache_read=112,217
turn N+1 first call: cache_creation= 60,635 cache_read= 53,896 ← system+tools only
turn N+2 first call: cache_creation= 65,755 cache_read= 53,896 ← same floor again
```
`53,896` is this deployment's `system` + tools prefix. Across all fallen turns the value clusters at 53,305–54,058 — always that floor, never a partial match further into the conversation.
Measured over 3,189 resume boundaries (production traffic, 2026-07-19 → 09-02):
| resume opens turn | falls to the floor |
|---|---|
| **turn 2** | **0 / 360 = 0.0%** |
| **turn 3** | **140 / 210 = 66.7%** |
| turn 4 | 75 / 147 = 51.0% |
| turn 5 | 34 / 101 = 33.7% |
| turn 6+ | 26–38% |
Turn 2 never falls, 360 for 360. Turn 3 falls two times out of three. Turn 2's resume looks at a conversation of ~55k (median); turn 3's looks at ~104k.
Conversation size is the sharp variable (idle gap >60s held constant):
| conversation size at the previous turn's end | falls |
|---|---|
| 40–82k | **0.0%** (0/173, five consecutive bins) |
| 82–88k | 9.7% |
| 88–95k | 24.2% |
| 95k+ | 30–31% |
Nothing at all below ~82k. Idle gap matters too but has no threshold — it rises gradually from ~30s and plateaus at 20–26% within a few minutes. (A `>60 min` bin sits at 95%, but that is the 1-hour TTL expiring as designed and is excluded from everything above.)
## Why this looks like #974 / #1120 rather than a host-side mistake
**#974** — "Bundled CLI never reads prompt cache in multi-turn headless runs" — is the same shape: headless, multi-turn, CLI writes every turn and reads nothing, while the plain `anthropic` SDK over the same endpoint caches correctly. It was fixed server-side on 2026-07-07, confirmed healthy even on the originally-broken bundled CLI without upgrading.
I ran the equivalent control here. Same region, same account, same model, a byte-identical 117k prefix with `cache_control: {type:"ephemeral", ttl:"1h"}` sent directly through `bedrock-runtime:InvokeModel`, no Claude Code involved:
```
t=0s cache_creation=117,017 cache_read= 0
t=20s cache_creation= 0 cache_read=117,017
t=90s cache_creation= 0 cache_read=117,017
t=5m cache_creation= 0 cache_read=117,017
t=10m cache_creation= 0 cache_read=117,017
```
Full hit every time, well past both thresholds. The provider honours the 1h TTL exactly as asked, so this is above the API layer.
**#1120** — "SessionStart hook context and CLI reminder blocks are emitted after the final `cache_control` breakpoint" — closed 2026-07-16. Its core observation was that *"turn 1's cache write is never read back, since the trailing message shifts the layout between turns."* That is the same mechanism I can see indirectly here: **on a fallen turn the rebuilt request is smaller than the version that was cached.** `cache_creation − (expected delta)` has a median of **−6.0k** tokens (p25 −17.7k). The prefix no longer matches because content is missing from it, not because a cache entry expired.
Both of those were fixed before the window measured here. This deployment's first bad build is **2.1.228**, which shipped in `claude-agent-sdk` **v0.2.136** (2026-08-11), i.e. a month after #1120 closed.
## Version boundary
Same day, same workload, three CLI versions alive at once (rolling deploys leave older containers running):
```
2026-08-12 2.1.207 0 / 29 = 0.0%
2026-08-12 2.1.226 0 / 15 = 0.0%
2026-08-12 2.1.228 7 / 27 = 25.9%
```
Holding both trigger conditions constant across the whole window:
| | ≤ 2.1.227 | 2.1.228+ |
|---|---|---|
| conversation >82k AND gap >60s | **11.5%** (110/954) | **48.5%** (431/889) |
So this is a **pre-existing fault that got ~4x more frequent**, not a clean regression — it was already happening at 11.5% before 2.1.228. Still present on 2.1.252.
Given #974 was ultimately fixed server-side while the same bundled CLI stayed in place, I can't fully separate "the 2.1.228 binary" from "a rollout that happened around the same time". The version correlation is what the data shows; the causal claim is weaker than that.
## Ruled out on the host side
Each measured, not assumed:
- **Transport of the transcript.** Floor rate is 44.0% when the container is new and the transcript comes from S3 (n=432), 43.7% when the container is reused and the local file is read directly (n=158). I also reimplemented the whole thing as a `SessionStore` adapter (S3-backed, passes the bundled `run_session_store_conformance`) and re-ran the same workload: same rate, total `cache_creation` **8% higher**, plus three partial hits (62.1%, 67.7%, 48.4%) the file route never produced. Both routes go through the same rebuild.
- **System prompt stability.** This host passes a custom `system_prompt` string, so the `claude_code` preset's per-session sections (cwd, platform, OS version, git status) are not in the prompt at all — `exclude_dynamic_sections` is neither applicable nor needed here. Confirmed empirically: the floor value is constant at 53,305–54,058 across every fallen turn, which can only happen if the prefix is byte-identical every time.
- **Concurrency.** No two sessions ever share a container within ±5 min (0 of 592).
- **Compaction / microcompact.** 0 occurrences in 60 sampled sessions; no `isCompactSummary`, no sidechains.
- **TTL.** `cache_creation` is 100% `ephemeral_1h_input_tokens`. Cache entries are alive; they are being missed, not expiring.
- **`totalTokensReminder`** (claude-code#90018). The `` attachment only starts appearing in these transcripts on 8/17, but the floor rate is already 23–34% on 8/13–8/16. Splitting on that date gives 23.8% (no reminder) vs 26.1% (reminder). A local A/B with it off changed nothing.
- **Application code.** The deploy that carried 2.1.228 also carried unrelated changes of ours; stratifying by conversation size makes that effect vanish while the version effect survives.
## Environment
- `claude-agent-sdk` (Python), headless, **one process per turn**, `resume=`
- Amazon Bedrock, ap-northeast-1, Application Inference Profiles (`claude-opus-5` and `claude-sonnet-5`, both affected at the same rate)
- Linux container (Bedrock AgentCore Runtime)
- 36 in-process SDK MCP tools, `ENABLE_TOOL_SEARCH=false` (tool definitions ride the cached prefix)
- `ENABLE_PROMPT_CACHING_1H=1`
- `system_prompt` = custom string; `setting_sources=["project"]`; `skills=[...]`; one `Stop` hook and one `PreToolUse` hook (the latter returns `additionalContext` only on a time-budget warning — 11 occurrences in the whole window)
- First bad: bundled CLI **2.1.228** (SDK v0.2.136). Last good: **2.1.227** (SDK v0.2.135) at 11.5%. Current: 2.1.252, still affected.
## Impact
**~$534/month** on this workload, at Opus cache-write rates. Per-turn cost went from $0.644 to $1.545 while the number of turns *fell* 10%. It is the dominant term in a +63% month-over-month bill.
## What I could not determine
I have not captured the outgoing request bodies, so "the rebuilt request is smaller" is inferred from `message.usage` arithmetic rather than observed directly. What content goes missing, and why the ~82k token floor exists at all, are both open. I'd be glad to run an instrumented build or a specific probe if that helps.
Filed on the CLI side as anthropics/claude-code#91151 with the same data. Raising it here because the failure only appears in the SDK's one-process-per-turn resume pattern, which is not something an interactive CLI session exercises.
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.