anomalyco / anomalyco/opencode

Explicit cache breakpoints are applied only to Anthropic-family models; every other family relies on implicit caching

Open
#48,246 3 comments 0 reactions 1 assignee View on GitHub

@jlongster is already working on this.

Since Sep 9, 2026.

Dominant language
TypeScript
Stars
209k
Forks
27.5k
PR merge metrics
PR metrics pending

Description

Description

Explicit cache breakpoints only ever reach Anthropic-family models, so every other family runs on implicit prefix caching with no advancing anchor.

message() at packages/opencode/src/provider/transform.ts:471-484 calls applyCaching() behind a family gate (providerID === "anthropic", api.id/model.id containing anthropic or claude, npm @ai-sdk/anthropic or @ai-sdk/alibaba). A model on @ai-sdk/openai matches none of it, so the function never runs. The openrouter/bedrock/copilot keys inside its map are for Claude routed through those gateways, not for those providers' own models.

applyCaching() puts a breakpoint on the last two non-system messages, which moves forward each turn. Nothing else gets that. Once anything mutates the array mid-conversation, reuse freezes at a fixed offset for the rest of the session:

turn    input    cached
 #18    2,750    95,104     healthy
 #19   95,691     3,712     break
 #26   70,278    38,912     prompt grew 9k, cached frozen

Turn 21 ran 12 seconds after turn 20 and reused 38,912 of a ~100k prefix just written, so it isn't TTL. Measured over 10 days, the split follows model family rather than provider — ~32,000 Anthropic-family turns at 100.0% reuse, versus gpt-6-astra 92.1%, gpt-5.6-sol 64.9%, github-copilot/gpt-5.4-mini 80.1%, github-copilot/gemini-3.5-flash 91.3%. Astra alone re-sent ~39M tokens.

Note this is breakpoint placement, not cache keys — promptCacheKey = sessionID is already set at :1321.

@ai-sdk/openai@3.0.88 (pinned at v1.18.30) already supports the field:

function getPromptCacheBreakpoint(providerOptions) {
  return providerOptions?.openai?.promptCacheBreakpoint;
}

It has zero references in this repo. Part-level providerOptions already survive into content blocks via mapProviderOptions() at :447, so the placement path exists on the Responses API that Astra uses.

Suggested fix, in two parts, since they live in different places:

  • Breakpoint — set promptCacheBreakpoint on content parts of the same trailing messages applyCaching() targets. That's the message path.
  • Modeprompt_cache_options is serialized model-level from openaiOptions.promptCacheOptions, so mode: "explicit" belongs in transform.options(), the same function already setting promptCacheKey at :1321. A message-path change alone sets breakpoints and leaves mode unset.

I'd avoid widening the existing applyCaching() gate — it also decides which families get cache_control blocks injected, and admitting @ai-sdk/openai risks sending those to openai-compatible endpoints that reject them.

Scope note: this covers the openai providerOptions namespace, which sdkKey() also maps @ai-sdk/amazon-bedrock/mantle onto. It does not reach Copilot — sdkKey("@ai-sdk/github-copilot") returns "copilot", so github-copilot/gpt-5.4-mini would stay at 80.1% and needs its own path. Worth treating as a separate change.

Related: #44727 (same mechanism, scoped to post-/compact), #43507, #39679.

Plugins

@cortexkit/opencode-anthropic-auth@1.22.0, @cortexkit/opencode-openai-auth@0.7.1, oh-my-opencode-slim@2.2.11, @cortexkit/opencode-magic-context@0.41.4, @cortexkit/aft-opencode@0.55.1, opencode-copilot-delegate@0.12.1, @fro.bot/systematic@3.16.5

OpenCode version

1.18.29, confirmed identical at v1.18.30

Steps to reproduce
  1. Run a long session on any @ai-sdk/openai model.
  2. Let the array be mutated mid-conversation (plugin injection or a compaction).
  3. Compare usage.prompt_tokens_details.cached_tokens to total prompt tokens per turn.
  4. Cached tokens stop tracking the prompt and pin at a fixed value, persisting across sub-minute gaps.
  5. Repeat on an Anthropic model — reuse tracks continuously and recovers after compaction.
Screenshot and/or share link

n/a

Operating System

macOS 15.7.9 (arm64)

Terminal

Ghostty

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.