[BUG] `appendSystemPrompt` with per-turn content silently destroys provider prompt caching
Personne n'a encore pris cette issue.
Évaluation
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Accessibilité débutants
- 48/100
- Type d'issue
- Bug
- Clarté
- Plutôt claire
- Activité
- Active
- Domaine
- cli, developer-experience, performance
Piste de recherche
Commencez par les surfaces du harness appendSystemPrompt et transformContext, puis lisez la documentation de ModApi et examinez l’observer model_request_end décrit dans l’issue. Reproduisez le comportement avec la configuration à deux mods et comparez cacheReadTokens d’un tour à l’autre. Le travail est terminé lorsque le contrat de stabilité, les indications concernant la tail-injection et l’avertissement proposé en dev-mode ont été traités ou que leur périmètre a été défini.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Description
[BUG] appendSystemPrompt with per-turn content silently destroys provider prompt caching
Summary
Custom mods that return per-turn dynamic content from the appendSystemPrompt hook
silently destroy provider prompt caching. Everything appendSystemPrompt emits is
prepended as a prefix before the whole conversation, and provider prompt caches
(Anthropic / OpenAI / DeepSeek-style prefix caching) key on the longest common
prefix — so one changed character invalidates the cache and forces the provider to
re-process the entire history (system prompt + every prior turn) on that turn.
There is no mod-side cache API, and nothing in the ModApi documents or enforces
that appendSystemPrompt output must be byte-stable across turns. It is the
natural hook to reach for when a mod "just wants to add context" — and it is the
worst possible place for content that changes.
A related, separate failure mode: splicing dynamic messages one slot too deep in
transformContext (at length - 2 instead of length - 1) spreads a cache miss
across two trailing messages instead of confining it to the final one.
Expected Behavior
appendSystemPromptis documented as a static or one-shot hook: output must
be byte-identical across turns (or stable per discrete state), or injected once
behind a guard flag and then returnundefined.- Per-turn dynamic context is supported through a first-class, cache-safe mechanism
(today the de-facto pattern is tail-append viatransformContext). - In dev mode, the harness warns when a registered
appendSystemPromptoutput
changes between turns, so mod authors catch cache-breaking prompt churn early.
Actual Behavior
- The
appendSystemPromptcontract is silent on cache stability; mod authors
reasonably use it for per-turn content (round counters, threshold advisories)
and get near-zerocacheReadTokenswith no diagnostic. - Any single character change in the system-prompt prefix invalidates the cached
prefix for the entire conversation history on that turn. - Incorrect message splice depth in
transformContext(e.g.length - 2instead
oflength - 1) spreads a cache miss across two trailing messages instead of
confining it to the final one — even when the system prompt itself is stable.
Steps to reproduce the issue
- Install or author a mod whose
appendSystemPrompthook returns content that
changes each turn (e.g. embedRound N/Min a briefing prompt, or flip
advisory text at a threshold crossing). - Run a multi-turn session against a provider that supports prefix caching.
- Observe usage on
model_request_end— e.g. via a pure observer mod that reads
cacheReadTokens/cacheWriteTokensand registers no prompt hooks. - Compare against a byte-stable system prompt (static / one-shot
appendSystemPromptoutput, dynamic content appended to the message tail
viatransformContextinstead).
Result: system-prompt churn correlates with near-zero cache reads; byte-stable
system prompt + tail injection yields consistent cache hits.
Optional related repro: in transformContext, splice a dynamic recall message at
messages.length - 2 instead of length - 1 and observe the cache miss span two
trailing messages rather than one.
Command Code Version
0.1.2 (Desktop)
Operating System
Windows
Terminal/IDE
Command Code Desktop / CLI
Shell
PowerShell
Session file (optional)
No response
Fix prompt (optional)
Document and harden the mod caching contract:
- Document
appendSystemPrompt: output must be byte-identical across turns
(or stable per discrete state), or one-shot guarded; per-turn content belongs
on the message tail, not the system prefix. - Consider a first-class cache-safe per-turn injection API — standardize the
tail-append pattern mod authors already use intransformContext. - Add a dev-mode diagnostic: serialize-and-compare
appendSystemPromptoutput
per turn; warn when it changes.
Relevant harness surface: appendSystemPrompt, transformContext, ModApi docs,
and optional dev-mode diagnostics. Verify with a two-mod setup: one that injects
dynamic system text (should warn / miss cache) and one observer that only reads
cacheReadTokens on model_request_end.
Additional context
Evidence from our mod suite (all occurrences fixed in the mods on 2026-08-23,
not the harness — see command-code-mods CHANGELOG 1.1.0):
- Per-round counter in a briefing prompt (
command-center). A plan-briefing
state machine embeddedRound N/Min its BRIEFING system prompt. It changed
every round, forcing a full-history re-process per round (up tomaxRounds
times per briefing). Fixed by removing the counter fromappendSystemPrompt
and tail-injecting it viatransformContext. - Advisory warnings that change at threshold crossings (
quality-guards).
Four guardrail warnings (drift, test-budget, token-budget, run-length) lived
inappendSystemPrompt. Their text changed at every warning crossing, so the
system prompt stopped being a stable prefix and forced full-history
re-processes. Fixed by moving the same thresholds/counters to a
transformContexttail injection. - Recall injection splicing one message too deep (
learn-loop,memory-bank).
Separate fromappendSystemPrompt: recall rodetransformContextbut was
spliced atlength - 2instead oflength - 1, spreading a cache miss across
two trailing messages instead of confining it to the final one.
A pure observer mod (cache-tracker, added afterward) confirmed the correlation
end-to-end: system-prompt churn → near-zero cache reads; byte-stable system prompt
→ consistent cache hits. Measured interactive sessions on DeepSeek V4 after the
fix showed ~48.7% hit rate over 25 turns (see suite README).
Root cause
- No mod-side cache API; cache performance is entirely at the mercy of provider
prefix caching. appendSystemPromptcontract undocumented on stability requirements.- No diagnostic when registered prompt output changes between turns.
- No guidance that
transformContextsplices should stay at the tail
(length - 1) to confine misses.
Workaround shipped in affected mods
appendSystemPromptcarries only static / one-shot / per-state-stable content
(e.g. memory-bank's session-start digest is one-shot behind a guard flag;
command-center's BRIEFING/COMPILING/REVIEW prompts are byte-stable within each
state).- Per-turn dynamic content (round counter, advisories) moved to
transformContext
tail injection so the cached prefix stays byte-identical. - Recall/injection splices at
length - 1to confine cache misses to a single
final message. - Cache observability lives in a separate observer mod that registers no prompt
hooks, so instrumentation never changes the measured value.
Mods affected: command-center, quality-guards, learn-loop, memory-bank
- Langage dominant
- Aucune donnée de langage
- Étoiles
- 4k
- Forks
- 350
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Autres issues de CommandCodeAI/command-code
-
Difficulté 2/5 1-3 heures Accessibilité débutants 68/100
CommandCodeAI/command-code#855 ·
-
Difficulté 2/5 1-3 heures Accessibilité débutants 78/100
CommandCodeAI/command-code#841 · 1 commentaire ·
-
Difficulté 2/5 1-3 heures Accessibilité débutants 68/100
CommandCodeAI/command-code#655 · 1 commentaire ·
-
Difficulté 2/5 1-3 heures Accessibilité débutants 68/100
CommandCodeAI/command-code#608 ·
-
Difficulté 3/5 1-2 jours Accessibilité débutants 70/100
CommandCodeAI/command-code#893 ·
Toutes les issues de CommandCodeAI/command-code
Issues similaires
-
Difficulté 1/5 Moins d'une heure Accessibilité débutants 92/100
milvus-io/birdwatcher#545 ·
-
Difficulté 2/5 1-3 heures Accessibilité débutants 88/100
-
area:tools bug good first issue help wanted priority:P2
Difficulté 2/5 1-3 heures Accessibilité débutants 90/100
TaewoooPark/Motifcode#14 ·
-
bug
Difficulté 2/5 1-3 heures Accessibilité débutants 84/100
newrelic-experimental/preflight#793 · 1 commentaire ·
-
Difficulté 2/5 1-3 heures Accessibilité débutants 88/100
caddyserver/caddy#8046 ·