0xPlaygrounds / 0xPlaygrounds/rig

feat: manually placed prompt-cache breakpoints for Anthropic

Offen
#2,168 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Rust
Sterne
8.6k
Forks
959
Ø Merge
4 Std. 32 Min.
Gemergte PRs (30 T.)
117

Beschreibung

- [x] I have looked for existing issues (including closed) about this

## Feature Request

Allow callers to place an Anthropic prompt-cache breakpoint (`cache_control`) at a chosen position in the message history, instead of only the fixed positions used by `with_prompt_caching` / `with_automatic_caching`.

### Motivation

The `CacheControl` type and per-block `cache_control` wire fields already exist in the provider, but request conversion always emits `cache_control: None` for content blocks — the only way to get markers is the model-level flags, which pin them to the system prompt and the latest message. Callers with a large stable prefix mid-history (a big document, a long tool manifest, an agent scaffold) can't mark it, so they can't use [Anthropic prompt caching](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) effectively from rig. We've been running this internally as a patch and are offering it upstream.

### Proposal

A documented `additional_params` key, `ANTHROPIC_CACHE_CONTROL_KEY` (`"anthropic_cache_control"`), set on a generic `message::Text` / `Image` / `Document` block to a `CacheControl` payload. Request conversion extracts it onto the corresponding content block's `cache_control` field; an invalid payload under the key fails conversion. Marker presence is the opt-in — no model-level flag. This follows the existing pattern of per-block provider metadata in `additional_params` (like the Anthropic document `title` / `context` / `citations` keys).

PR to follow. **Design note:** we made some speculative API decisions here (the `additional_params` carrier, the public key const, interaction contract with `with_prompt_caching`) and are happy to redo it differently if you'd prefer another shape.

### Alternatives

- A first-class `cache_control`-like field on the generic message types — cleaner but leaks provider-specific structure into the provider-agnostic surface.
- Typed attach helpers (`anthropic_text_with_cache_control(...)`) — better ergonomics/validation but new API surface in a style the library doesn't use elsewhere; we prototyped this and backed it out in favor of the raw-key pattern.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

The issue involves adding Anthropic prompt-cache breakpoints via an `additional_params` key. Start by examining the request conversion logic for Anthropic messages, likely in a provider-specific module. Look for existing handling of `additional_params` (like document `title`/`context`/`citations`). The PR mentioned may provide context; check linked discussions. Implement extraction of the `anthropic_cache_control` key onto the content block's `cache_control` field. Ensure validation of the `CacheControl` payload. Test with the existing Anthropic integration tests to verify the marker is correctly set in the wire format.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
rust
Bereich
ai-infra-agents
Issue-Typ
Feature
Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Aktivitätsstatus
Ruhig
Klarheit
Klar beschrieben
Anfängerfreundlichkeit
55/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.