anthropics / anthropics/claude-code

Per-session scratchpad UUID in the system prompt is the only cross-session prompt diff, invalidating the cached prefix on every new session

Ouverte
#92,524 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
area:core area:cost bug has repro platform:macos
Langage dominant
Python
Étoiles
145k
Forks
23.1k
Métriques de merge des PR
Métriques de PR en attente

Description

## Summary

The system prompt embeds the scratchpad directory path, which contains a
**per-session UUID**. That UUID is the only thing that differs between two
otherwise byte-identical requests from consecutive sessions in the same project —
and because it sits inside the cached prefix, everything after it is invalidated
on every new session.

I captured two real request bodies from two consecutive `claude -p "hi"` runs in
the same directory and diffed them:

| block | size | across sessions |
|---|---|---|
| `tools` (137 MCP tools) | 223,596 chars | **byte-identical, same order** |
| `messages` | 50,534 chars | **byte-identical** |
| `system` | 7,483 chars | differs at exactly one place |

The single difference:

```
...-scratchpad/fc9aab5b-da05-4cc7-bbe2-410a1c472151/scratchpad
...-scratchpad/88966ec9-2cb4-4872-bc6b-624b66257462/scratchpad
```

from this block of the system prompt:

```
IMPORTANT: Always use this scratchpad directory for temporary files ...
`/private/tmp/claude-501///scratchpad`
```

The same UUID also appears in `metadata.user_id`, but that field is not part of
the prompt and is harmless.

## Measured cost

Against a local model (so I can read `usage.input_tokens` directly and control
the cache), 65,303-token prompt, warm model, idle M5 Max:

| request | wall | `input_tokens` |
|---|---|---|
| cold | 104.3 s | 65,303 |
| identical prompt again | 0.3 s | 5 |
| **next session — only the UUID differs** | **30.6 s** | **16,157** |
| same request with the UUID normalised | **0.4 s** | **5** |

Normalising that one UUID in a proxy turns a 30.6 s re-prefill into 0.4 s.
End-to-end with MCP enabled, three consecutive real sessions went from
131.7 s → 17.3 s → 8.3 s.

## Why this likely matters for hosted use too

I measured locally because that is where I can see the cache directly. But the
mechanism is not local-specific: the volatile UUID sits in `system`, ahead of the
tool definitions and the conversation, so it is inside whatever prefix a
`cache_control` breakpoint covers. If that is how the hosted path is laid out,
every new session in a project pays a full system+tools cache write instead of a
read, for a value that has no bearing on the model's output. I have not measured
the hosted side and am not claiming a number for it — but it seems worth checking,
since the fix is cheap either way.

## Suggested fixes, cheapest first

1. **Move the scratchpad path out of the cached prefix** — put it in the last
user turn (or a system-reminder near the end) rather than in the stable system
block. Zero behaviour change, and the prefix becomes session-invariant.
2. **Make the path derivable and stable** — e.g. key it on the project path plus a
stable session ordinal, or reuse one scratchpad per project. The directory is
temporary storage; its name does not need to be unique per session for the
model to use it correctly.
3. If a unique path per session is genuinely required, **place the volatile
segment as late as possible** in the prompt so that the tool definitions
(223 KB here, by far the largest block) stay in the shared prefix.

## Related

- #92033 — mid-conversation tool/MCP list changes invalidating the prefix. Same
cache, different trigger; that one is within a session, this one is across
sessions.
- #90953 — a "why did this cache miss" diagnostic would have found this in
minutes instead of a byte-level diff of two captured request bodies.

## Environment

Claude Code 2.1.260, macOS 26.5, M5 Max. Reproduced by pointing
`ANTHROPIC_BASE_URL` at a local capture server and diffing the two request
bodies; no special configuration needed to see the UUID differ.

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Piste de recherche

Reproduce the issue with two consecutive `claude -p "hi"` runs in the same directory and compare captured request bodies through `ANTHROPIC_BASE_URL`. Trace where the system prompt and scratchpad path are assembled, then verify that the UUID no longer invalidates the stable prefix while session-specific scratchpad behavior remains intact.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Domaine
cli, performance
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Active
Clarté
Plutôt claire
Accessibilité débutants
45/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.