[BUG] Prompt caching breaks when a plugin appends a message

Description

Prompt caching stops working when a plugin appends a message to the request. Nothing errors. The only symptom is a rising cache_creation_input_tokens count.

Caching works by marking a point in the conversation. Everything before that point is re-sent cheaply on the next turn instead of being paid for again. OpenCode places two of these markers — cache breakpoints — on the last two non-system messages, in applyCaching at packages/opencode/src/provider/transform.ts:361:

const final = msgs.filter((msg) => msg.role !== "system").slice(-2)

Plugins append messages through experimental.chat.messages.transform. What they typically append is a status notice, rebuilt on every request. When such a notice is last in the array it receives a breakpoint, and that breakpoint can never produce a cache hit: Anthropic caching matches on a prefix, and fresh conversation is inserted ahead of the notice on every turn, so the cached region is no longer a prefix of the new request. Cache reads stop advancing.

Making the notice byte-identical across requests does not help. The problem is its position, not its content.

With two such notices, both breakpoints land on regenerated content and no cached prefix survives to the next turn.

Measured over four consecutive requests in one session: cache reads held at exactly 300,150 while cache writes climbed 9,796 → 15,382 → 18,805 → 25,074. All 69,057 written tokens were paid for and never read back. The 200,600-token cache rebuild that followed also failed to persist. A cache read costs roughly a tenth of a cache write, so rewriting that prefix each turn costs about ten times what reading it would have.

OpenCode reproduces a smaller version of this without any plugin involved. On the last step of a turn it appends MAX_STEPS_PROMPT to the end of the message array (session/prompt.ts:1281). That message receives a breakpoint that can never produce a cache hit. Losing one breakpoint on last steps is tolerable in isolation, but it also means a single plugin notice is enough to lose both.

The consequence for plugin authors is that the cost is invisible and unavoidable. There is no way to append a message through the supported hook without risking the cache, and no signal that it has happened.

The native path in packages/llm/src/cache-policy.ts appears to have the same shape — latest-user-message selects by role alone, and { tail: N } is a fixed positional window — but we have not measured that route, so this report is scoped to the one we have.

We have a fix running in a local build and will open a PR against this issue. It is not a small patch: dev currently has no way to determine which messages a plugin appended, so some plumbing is required first. That is described in the PR.

Plugins

custom

OpenCode version

1.17.11 custom build

Steps to reproduce

1. Use a model with prompt caching enabled. Ours is Anthropic.
2. Load a plugin whose experimental.chat.messages.transform hook appends two messages to output.messages. Keep their text byte-identical across requests, so text instability is excluded as a cause.
3. Run a session long enough to build a large cached prefix, then continue for several more turns.
4. Observe cache_read_input_tokens and cache_creation_input_tokens on the responses. Reads stop advancing and hold at a fixed value while creation grows on every request.

One awkward asymmetry: the simplest reproduction only demonstrates the mild form. Stock opencode appends at most one message (MAX_STEPS_PROMPT, and only on the last step), and a single appended message is survivable — one of two breakpoints is lost. Two appended messages are what eliminate the cached prefix entirely, and reaching two requires a plugin. We are not aware of a way to demonstrate the severe case without one.

Screenshot and/or share link

No response

Operating System

Windows 10

Terminal

Windows Terminal (Powershell)