Lean into provider-native caching (explicit Gemini context caching, docs, visibility)

Motivation

Client-side response caches (LangChain's set_llm_cache, LiteLLM's caching layer) date from early 2023, before Anthropic prompt caching (Aug 2024), OpenAI automatic prompt caching (Oct 2024), or Gemini context caching existed. The providers now solve the underlying cost/latency problem better, and chatlas already leans into that harder than most frameworks:

• Anthropic/Bedrock: cache="5m" on by default, with cache_control injection and cache-write cost multipliers (1.25×/2×) reflected in get_cost() (_provider_anthropic.py).
• OpenAI: automatic server-side caching; cached_tokens extracted and priced (_provider_openai.py).
• Google: implicit-cache token counts tracked (_provider_google.py).
• get_tokens() / get_cost() expose cached_input as a first-class field; prior art in #103, #133, #158, #243 (all closed).

What remains is closing the gaps, not adding a new cache layer.

Proposed work

1. Explicit Gemini context caching — the one concrete API gap. Support Google's explicit caching (client.caches.create(...) with a TTL, then reference the cached content across calls). This dovetails with the new Chat.files accessor: same "register once, reference cheaply across turns" model. Whether it surfaces as a chat.files-like accessor, a content type (cf. #136 content_ephemeral()), or a ChatGoogle() parameter is the main design question.
2. A caching guide in the docs — the current defaults are good but invisible. Cover: what each provider does automatically, how to read cached_input from get_tokens() to check your hit rate, Anthropic TTL tradeoffs ("5m" vs "1h" write multipliers), and how to keep prompts cache-friendly (stable prefix; don't churn the system prompt or tool set mid-conversation).
3. Small parameter exposure — e.g. OpenAI's prompt_cache_key routing hint for improving hit rates. Low priority for chatlas's typical single-process user, but cheap.
4. Cross-issue design note — prefix caches reward immutable history, so history compaction (#356) and mutable system prompts are cache-busting operations. The #356 design should account for the one-time cost cliff after a compaction.

Non-goal: client-side response cache

A LangChain-style client-side response cache is out of scope. Its one honest use case — replaying an identical request without paying again (prompt caching discounts input tokens but never eliminates the call or the output tokens) — is better served elsewhere:

• Notebook/script re-runs: that's a persistence problem — reload the earlier conversation (#352 chat.save()/Chat.load()) instead of re-asking the model.
• Tests/CI: VCR-style request recording (what chatlas's own test suite uses via pytest-recording) is purpose-built for this.
• Treating nondeterministic calls as cacheable pure functions also has real surprise potential (a cache silently freezes responses that users expect to vary).

Part of the July 2026 competitive-review workstreams (#360).

Start with _provider_google.py and the new Chat.files accessor, then compare cache handling in _provider_anthropic.py and _provider_openai.py. Clarify the API design for explicit Gemini context caching, the caching guide, and any small parameter exposure. Done means the agreed provider-native caching behavior and documentation cover cached_input, provider defaults, TTL tradeoffs, and cache-friendly prompts.