OpenRouter+Anthropic: prompt caching disabled, ~3-4x cost overhead

Summary

OpenAI-compat provider hardcodes CacheControl: false in capabilities. When OpenRouter routes Anthropic Claude models, no cache_control markers are injected, so Anthropic prompt caching never engages. Result: every call resends the full prompt at full input rate.

Evidence

7-day usage from usage_snapshots on a production deployment:

DeepSeek caches via the OpenRouter provider's own caching mechanism. Sonnet through OpenRouter gets nothing. Spans for model='anthropic/claude-sonnet-4.6' consistently show ~157k input tokens per call with empty metadata (no cache_read_input_tokens/cache_creation_input_tokens).

Cost impact: 46.5M × $3/M ≈ $140/week. With 80% cache hit (matching DeepSeek's ratio): ~$40/week. ~$100/week wasted per tenant. Per-call observed cost ~$0.23 average, ~$0.47 peak.

Root Cause

internal/providers/openai_config.go:99:

func (p *OpenAIProvider) Capabilities() ProviderCapabilities {
    return ProviderCapabilities{
        ...
        CacheControl:     false,  // hard-disabled for all openai-compat providers
        ...
    }
}

internal/providers/middleware_cache.go only injects OpenAI-native prompt_cache_key / prompt_cache_retention and explicitly bails for proxies via isOpenAINativeEndpoint.

internal/providers/anthropic.go:117 correctly sets CacheControl: true and internal/providers/anthropic_request.go injects cache_control: {type: ephemeral} on system block + last tool + recent messages — but only when the agent is bound to a native Anthropic provider, not OpenRouter.

OpenRouter supports the cache_control extension for Claude models in its OpenAI-compat schema (passthrough to Anthropic). The markers just need to be sent.

Suggested Fix

Two paths:

1. Conditional capability + injection in openai-compat path. When providerType == "openrouter" (or detected Anthropic upstream) AND model matches anthropic/* or claude*, set CacheControl: true and apply the same block-level cache markers used in anthropic_request.go (system text block + last tool + last 1–2 user messages).
2. Generic OpenAI-compat cache extension. Some OpenAI-compat backends (LiteLLM, OpenRouter) accept cache_control on message content blocks. Gate behind a per-provider setting (settings.cache_control_passthrough) so users can opt in for backends that support it.

Option 1 is targeted and matches what anthropic.go already does — least churn, highest impact.

Workaround

Configure a native Anthropic provider in the dashboard and re-point Claude-using agents to it. Native adapter handles caching automatically. Loses OpenRouter's load-balancing/failover, but cuts cost ~70%.

Reproduce

1. Configure OpenRouter as the only LLM provider.
2. Create an agent with provider=openrouter, model=anthropic/claude-sonnet-4.6.
3. Run any multi-turn workflow with stable system prompt + tools.
4. Check usage_snapshots — cache_read_tokens and cache_create_tokens will be 0 across all calls.

Affected Code

• internal/providers/openai_config.go:99 — capability flag
• internal/providers/openai_request.go — request body builder, no Anthropic-aware branch
• internal/providers/middleware_cache.go — bails on non-OpenAI endpoints

Start with internal/providers/openai_config.go, openai_request.go, and middleware_cache.go, then compare their behavior with internal/providers/anthropic.go and anthropic_request.go. Reproduce the OpenRouter Claude workflow and inspect usage_snapshots for cache reads and writes. Done means the selected supported path sends the expected cache markers and usage reflects caching without breaking other OpenAI-compatible providers.