anthropics / anthropics/claude-code

Emit --append-system-prompt as its own system block so a changed append does not invalidate the preset

Aperta
#91,518 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
area:core area:cost enhancement
Lingua principale
Python
Stelle
145k
Fork
23.1k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

### Summary

The CLI concatenates the `claude_code` preset and the caller's `--append-system-prompt` text into a **single** `system` content block, and places its `cache_control` mark at that block's end. Because a cache entry is keyed on a whole block, any change to the appended text invalidates the entry for the preset as well — so a caller whose appended content varies re-pays for the CLI's own system prompt every time.

Splitting the append into its own block, with a boundary in front of it, would confine that invalidation to the appended text.

### Measured

Captured with a local pass-through proxy in front of the CLI (2.1.258, Agent SDK 0.3.146, `claude-opus-5`, reproduced on `claude-haiku-4-5`). Every row is a `cache_control` block in a body the CLI actually sent:

| block | size | mark |
| --- | --- | --- |
| `system[0]` | 74 chars | — |
| `system[1]` | 94 chars | `1h` |
| `system[2]` | 27,723 chars — preset **+ append concatenated onto its tail** | `1h` |

The append began at char **27,642 of 27,723**, i.e. inside `system[2]` rather than as a block of its own.

Mark budget across rounds:

- **Round 1** — three CLI marks: `system[1]`, `system[2]`, last user content block.
- **Rounds 2+** — four: `system[1]`, `system[2]`, `tool_use`, `tool_result`. The CLI drops its last-message mark when it adds the tool pair.

All marks carry `ttl: 1h`.

### Why it costs

In a real workload the cached prefix is ~109k tokens, of which ~20k is appended content that changes occasionally (a set of instruction files that grows as a session progresses) and ~79k is the preset plus static appended material. Because it is all one block, changing the 20k rewrites the 79k with it, at cache-write price.

That inverts the economics of appending anything that is not perfectly static. In our measurements it takes roughly **four unchanged turns to pay back one changed turn** — so a caller whose appended content changes even occasionally is close to break-even on caching it at all, purely because of where the boundary falls.

### Request

Emit the appended system prompt as its own content block:

```
system[2] preset ← mark ← boundary here
system[3] append ← mark
```

Then a change to the append invalidates only the append; the lookback finds the entry at the end of `system[2]` and the preset is read from cache.

### The budget objection, addressed

This is a third `system` mark, and on a round carrying a tool call the CLI is already at four of the API's four. The request therefore includes **re-budgeting**: drop the mark on `system[1]` — 94 characters, which caches nothing worth having — and spend it on the preset/append boundary.

That is the same kind of budgeting the CLI already performs internally: it yields its last-message mark the moment it needs the `tool_use`/`tool_result` pair, as the capture above shows.

### Related but distinct

anthropics/claude-agent-sdk-typescript#432 asks for caller-supplied marks in streamed input to be counted against the budget. That would solve a caller's problem by handing it a mark to manage. This asks for the block boundary instead, which needs no mark from the caller at all and benefits everyone using `--append-system-prompt` with content that is not byte-static — a larger group than those placing their own marks.

#87487 (suppressing the daily `currentDate` injection) is the same family: both are about what a caller can keep stable inside a marked block.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Start at the CLI request-building path for --append-system-prompt and inspect how the claude_code preset, system content blocks, cache_control marks, and tool_use/tool_result pairs are assembled. Reproduce the captured request shape with a pass-through proxy, then verify that the preset and append are separate marked blocks without exceeding the API's four-mark budget.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
cli, performance
Tipo di issue
Funzionalità
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Attiva
Chiarezza
Abbastanza chiara
Idoneità per principianti
52/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.