anthropics / anthropics/claude-code

Mid-conversation tool/MCP list changes silently invalidate prompt cache, causing repeated full-price rewrites within minutes

Ouverte
#92,033 4 commentaires 2 réactions 0 personnes assignées Voir sur GitHub
area:core area:cost bug
Langage dominant
Python
Étoiles
145k
Forks
23.1k
Métriques de merge des PR
Métriques de PR en attente

Description

## Summary

In a single long-running Claude Code session (resumed intermittently over ~36 hours on the 5x Max plan), the prompt cache was repeatedly and fully invalidated by mid-conversation changes to the tool/MCP/agent list — not by idle/TTL expiry. This caused multiple full-price cache rewrites (each ~780K-860K tokens, billed at the 2x cache-write rate) within minutes of each other, including one pair only **1 minute apart**, burning what looked like an entire 5-hour session budget in under 10 minutes.

## Evidence

Transcript excerpt (timestamps UTC) from one session, `cw` = cache_creation_input_tokens, `cr` = cache_read_input_tokens:

| Time | cache write | cache read | Notes |
|---|---|---|---|
| 07:47:37 | 857,710 | 39,062 | full rewrite (expensive) |
| 07:47:50 | 3,032 | 896,772 | cache working correctly |
| 07:48:20 | 3,331 | 899,804 | cache working correctly |
| 07:48:34 | 1,168 | 903,135 | cache working correctly |
| **07:48:32** | — | — | `deferred_tools_delta`, `agent_listing_delta`, `mcp_instructions_delta`, 6× `edited_text_file` events fire in the transcript |
| 07:48:49 | 782,873 | 10,138 | full rewrite again, ~15s after the delta events and only 1 minute after the previous cheap turn |
| 07:48:55 | 996 | 793,011 | cache working correctly again |

The cache was demonstrably working correctly for three consecutive turns (reads climbing 897K → 900K → 903K, writes near-zero). It was then invalidated specifically at the moment new tool/agent/MCP-server definitions were loaded into the session (visible in the transcript as `deferred_tools_delta`, `agent_listing_delta`, and `mcp_instructions_delta` attachment events), forcing the very next turn to eat a full-price, 2x-weighted rewrite of the entire accumulated context (~800K+ tokens) — despite that context having just been cached a minute prior.

This is distinct from (and in addition to) cache expiry after idle gaps, which also repeatedly hit the same session across resumes spanning Sep 2–4, each time re-billing an ever-larger context (300K tokens growing to 860K+ tokens over the session's lifetime) at full 2x cache-write cost.

## Impact

- No warning is surfaced to the user before an action (e.g., a background `ToolSearch` call, a new MCP server connecting) triggers a full cache invalidation.
- No setting exists to prevent tool/MCP-list changes from invalidating the cache prefix mid-conversation.
- No setting exists to cap or warn on an unusually large single-turn cache-write.
- Combined with no context-compaction/summarization that persists in the cache, a single long-lived session (common in real usage, e.g., a standing project thread resumed over days) can silently accumulate an ever-growing context that gets fully, repeatedly rebilled at 2x cost every time anything touches the tool/instruction prefix — even seconds after the last rebuild.
- Concretely: this pattern consumed what appeared to be an entire 5-hour Max-plan session budget in under 10 minutes of actual interaction.

## Requested

1. Avoid invalidating the prompt cache prefix when only the *available* tool/agent/MCP list changes (e.g., via deferred tool loading) — ideally by structuring the system prompt so tool/instruction deltas don't sit ahead of the cached conversation history in the prefix.
2. Surface a warning (or at least a `/context`-visible flag) when an action is about to force a full cache rewrite, especially when the rewrite size is large relative to recent turns.
3. Expose a setting to control or defer tool/MCP-list changes mid-session so they don't retroactively invalidate an expensive, already-paid-for cache.
4. Consider a persisted, compacted-context mechanism so resuming a long session doesn't require re-paying full price for the entire accumulated history every time the cache is invalidated.

## Environment

- Claude Code desktop app, version 2.1.255
- Plan: Claude Max 5x

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Piste de recherche

The issue names no source files or tests; start by reproducing the reported cache behavior in Claude Code 2.1.255 and tracing the transcript events `deferred_tools_delta`, `agent_listing_delta`, and `mcp_instructions_delta`. Done means tool, agent, or MCP-list changes no longer silently trigger an avoidable full rewrite, with the requested warning or control behavior addressed.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Domaine
ai, devtools, performance
Type d'issue
Bug
Difficulté
5/5
Temps estimé
Plus d'une semaine
Activité
Active
Clarté
Plutôt claire
Accessibilité débutants
35/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.