anthropics / anthropics/claude-agent-sdk-python

Multi-turn headless resume falls back to the system+tools cache floor past ~82k tokens (CLI 2.1.228+, Bedrock) — same fingerprint as #974 / #1120

未關閉
#1,241 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
bug
主要語言
Python
星號
8.1k
分支
1.3k
PR 合併指標
PR 指標待擷取

描述

Multi-turn headless resume: the conversation cache falls back to the system+tools floor once the conversation passes ~82k tokens (bundled CLI 2.1.228+, Bedrock). Same fingerprint as #974 and #1120, both fixed and closed before this window.

## What happens

Headless, one process per turn, `ClaudeAgentOptions(resume=)`, transcript restored from S3 between turns (the "Move the session file" route from the sessions docs). On the first API call of a resumed turn, `cache_read_input_tokens` drops to exactly the `system` + tool-definitions length and the whole conversation is re-written.

```
turn N last call: cache_creation= 4,345 cache_read=112,217
turn N+1 first call: cache_creation= 60,635 cache_read= 53,896 ← system+tools only
turn N+2 first call: cache_creation= 65,755 cache_read= 53,896 ← same floor again
```

`53,896` is this deployment's `system` + tools prefix. Across all fallen turns the value clusters at 53,305–54,058 — always that floor, never a partial match further into the conversation.

Measured over 3,189 resume boundaries (production traffic, 2026-07-19 → 09-02):

| resume opens turn | falls to the floor |
|---|---|
| **turn 2** | **0 / 360 = 0.0%** |
| **turn 3** | **140 / 210 = 66.7%** |
| turn 4 | 75 / 147 = 51.0% |
| turn 5 | 34 / 101 = 33.7% |
| turn 6+ | 26–38% |

Turn 2 never falls, 360 for 360. Turn 3 falls two times out of three. Turn 2's resume looks at a conversation of ~55k (median); turn 3's looks at ~104k.

Conversation size is the sharp variable (idle gap >60s held constant):

| conversation size at the previous turn's end | falls |
|---|---|
| 40–82k | **0.0%** (0/173, five consecutive bins) |
| 82–88k | 9.7% |
| 88–95k | 24.2% |
| 95k+ | 30–31% |

Nothing at all below ~82k. Idle gap matters too but has no threshold — it rises gradually from ~30s and plateaus at 20–26% within a few minutes. (A `>60 min` bin sits at 95%, but that is the 1-hour TTL expiring as designed and is excluded from everything above.)

## Why this looks like #974 / #1120 rather than a host-side mistake

**#974** — "Bundled CLI never reads prompt cache in multi-turn headless runs" — is the same shape: headless, multi-turn, CLI writes every turn and reads nothing, while the plain `anthropic` SDK over the same endpoint caches correctly. It was fixed server-side on 2026-07-07, confirmed healthy even on the originally-broken bundled CLI without upgrading.

I ran the equivalent control here. Same region, same account, same model, a byte-identical 117k prefix with `cache_control: {type:"ephemeral", ttl:"1h"}` sent directly through `bedrock-runtime:InvokeModel`, no Claude Code involved:

```
t=0s cache_creation=117,017 cache_read= 0
t=20s cache_creation= 0 cache_read=117,017
t=90s cache_creation= 0 cache_read=117,017
t=5m cache_creation= 0 cache_read=117,017
t=10m cache_creation= 0 cache_read=117,017
```

Full hit every time, well past both thresholds. The provider honours the 1h TTL exactly as asked, so this is above the API layer.

**#1120** — "SessionStart hook context and CLI reminder blocks are emitted after the final `cache_control` breakpoint" — closed 2026-07-16. Its core observation was that *"turn 1's cache write is never read back, since the trailing message shifts the layout between turns."* That is the same mechanism I can see indirectly here: **on a fallen turn the rebuilt request is smaller than the version that was cached.** `cache_creation − (expected delta)` has a median of **−6.0k** tokens (p25 −17.7k). The prefix no longer matches because content is missing from it, not because a cache entry expired.

Both of those were fixed before the window measured here. This deployment's first bad build is **2.1.228**, which shipped in `claude-agent-sdk` **v0.2.136** (2026-08-11), i.e. a month after #1120 closed.

## Version boundary

Same day, same workload, three CLI versions alive at once (rolling deploys leave older containers running):

```
2026-08-12 2.1.207 0 / 29 = 0.0%
2026-08-12 2.1.226 0 / 15 = 0.0%
2026-08-12 2.1.228 7 / 27 = 25.9%
```

Holding both trigger conditions constant across the whole window:

| | ≤ 2.1.227 | 2.1.228+ |
|---|---|---|
| conversation >82k AND gap >60s | **11.5%** (110/954) | **48.5%** (431/889) |

So this is a **pre-existing fault that got ~4x more frequent**, not a clean regression — it was already happening at 11.5% before 2.1.228. Still present on 2.1.252.

Given #974 was ultimately fixed server-side while the same bundled CLI stayed in place, I can't fully separate "the 2.1.228 binary" from "a rollout that happened around the same time". The version correlation is what the data shows; the causal claim is weaker than that.

## Ruled out on the host side

Each measured, not assumed:

- **Transport of the transcript.** Floor rate is 44.0% when the container is new and the transcript comes from S3 (n=432), 43.7% when the container is reused and the local file is read directly (n=158). I also reimplemented the whole thing as a `SessionStore` adapter (S3-backed, passes the bundled `run_session_store_conformance`) and re-ran the same workload: same rate, total `cache_creation` **8% higher**, plus three partial hits (62.1%, 67.7%, 48.4%) the file route never produced. Both routes go through the same rebuild.
- **System prompt stability.** This host passes a custom `system_prompt` string, so the `claude_code` preset's per-session sections (cwd, platform, OS version, git status) are not in the prompt at all — `exclude_dynamic_sections` is neither applicable nor needed here. Confirmed empirically: the floor value is constant at 53,305–54,058 across every fallen turn, which can only happen if the prefix is byte-identical every time.
- **Concurrency.** No two sessions ever share a container within ±5 min (0 of 592).
- **Compaction / microcompact.** 0 occurrences in 60 sampled sessions; no `isCompactSummary`, no sidechains.
- **TTL.** `cache_creation` is 100% `ephemeral_1h_input_tokens`. Cache entries are alive; they are being missed, not expiring.
- **`totalTokensReminder`** (claude-code#90018). The `` attachment only starts appearing in these transcripts on 8/17, but the floor rate is already 23–34% on 8/13–8/16. Splitting on that date gives 23.8% (no reminder) vs 26.1% (reminder). A local A/B with it off changed nothing.
- **Application code.** The deploy that carried 2.1.228 also carried unrelated changes of ours; stratifying by conversation size makes that effect vanish while the version effect survives.

## Environment

- `claude-agent-sdk` (Python), headless, **one process per turn**, `resume=`
- Amazon Bedrock, ap-northeast-1, Application Inference Profiles (`claude-opus-5` and `claude-sonnet-5`, both affected at the same rate)
- Linux container (Bedrock AgentCore Runtime)
- 36 in-process SDK MCP tools, `ENABLE_TOOL_SEARCH=false` (tool definitions ride the cached prefix)
- `ENABLE_PROMPT_CACHING_1H=1`
- `system_prompt` = custom string; `setting_sources=["project"]`; `skills=[...]`; one `Stop` hook and one `PreToolUse` hook (the latter returns `additionalContext` only on a time-budget warning — 11 occurrences in the whole window)
- First bad: bundled CLI **2.1.228** (SDK v0.2.136). Last good: **2.1.227** (SDK v0.2.135) at 11.5%. Current: 2.1.252, still affected.

## Impact

**~$534/month** on this workload, at Opus cache-write rates. Per-turn cost went from $0.644 to $1.545 while the number of turns *fell* 10%. It is the dominant term in a +63% month-over-month bill.

## What I could not determine

I have not captured the outgoing request bodies, so "the rebuilt request is smaller" is inferred from `message.usage` arithmetic rather than observed directly. What content goes missing, and why the ~82k token floor exists at all, are both open. I'd be glad to run an instrumented build or a specific probe if that helps.

Filed on the CLI side as anthropics/claude-code#91151 with the same data. Raising it here because the failure only appears in the SDK's one-process-per-turn resume pattern, which is not something an interactive CLI session exercises.

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。