[loop-streaming] Ephemeral hook injections break prompt-cache prefix continuity — conversation history is re-written (1.25x) on every provider call

Summary

loop-streaming appends ephemeral hook injections (skills-visibility list ~5k tokens, status-context with a seconds-precision timestamp, todo reminder, etc.) by string-concatenating them into the last message's content on every provider request, without storing them in history (amplifier_module_loop_streaming/__init__.py:350 and :388, verified on main 2026-07-20).

Request N therefore caches a prefix ending in [...last_msg + injection_N], while request N+1 sends [...last_msg] (no injection) + new messages + injection_N+1. The prefix diverges exactly where the previous request's injection was baked in, so with Anthropic prompt caching conversation history never cache-hits beyond the initial system+tools prefix — it is re-written at cache-write rates (1.25x input) on every single call, growing each time.

Evidence (real session, claude-fable-5, from events.jsonl usage fields)

call 1:  cache_write=263,338  cache_read=0          <- session start
call 2:  cache_write=9,658    cache_read=254,421
call 3:  cache_write=10,750   cache_read=254,421
call 5:  cache_write=13,046   cache_read=254,421
...
call 12: cache_write=25,305   cache_read=254,421
(>5min gap) cache_write=281,993 cache_read=null     <- full re-write after TTL expiry
call 14: cache_write=28,944   cache_read=254,421

cache_read is frozen at the initial prefix forever; cache_write grows monotonically. 1.7M tokens cache-written across 27 calls in one session. Org-wide July 1–20: 83% of total API cost ($6,191 of $7,474) was input_cache_write_5m, with a cache read/write ratio of 2.0 (a healthy agentic workload is 5–10x+).

Suggested fix (two small coordinated changes)

1. loop-streaming: append the ephemeral injection as a separate trailing message instead of concatenating into the last real message.
2. provider-anthropic: place cache breakpoints on the last TWO messages (Anthropic allows 4 breakpoints; only 3 are used today — _apply_message_cache_control at __init__.py:3722). The last real message then remains a stable cache hit-point across requests, and the ephemeral tail becomes cheap uncached input (~5–7k tokens at 1x) instead of forcing a full-history re-write.

With both in place, conversation history caches incrementally as designed; per-call cache_write drops from tens of thousands of tokens (growing) to just the actual increment.

Happy to submit PRs for both if given the nod — I have the fix sketched and the measurement harness to verify before/after.

Environment

• Amplifier via uv tool install, macOS arm64
• amplifier-module-loop-streaming @ main (2e7232a-era cache), amplifier-module-provider-anthropic @ main
• Related observation: skills-visibility hook injects the full skills list (~5k tokens) into every provider request by default; capping visibility.max_skills_visible mitigates the tail cost but not the prefix-busting.

Start in amplifier_module_loop_streaming/init.py at lines 350 and 388, then inspect provider-anthropic init.py at line 3722 and the referenced _apply_message_cache_control path. Use the issue's measurement harness and events.jsonl usage fields to compare cache behavior before and after. Done means ephemeral injections no longer rewrite conversation history and cache_write reflects only the actual increment.