charmbracelet / charmbracelet/fantasy
Agent API: no way to place a cache_control breakpoint inside the system prompt
- Dominant language
- Go
- Stars
- 987
- Forks
- 137
- Avg merge
- 14h 41m
- Merged PRs (30d)
- 12
Description
### Summary
The Anthropic provider already supports per-part `cache_control` everywhere — including system blocks: `toPrompt` in `providers/anthropic/anthropic.go` reads `GetCacheControl(part.Options())` for each system `TextPart` (falling back to `msg.ProviderOptions` on the last part), so a multi-part system message can pin a cache breakpoint exactly where the stable prefix ends.
But that capability is unreachable through the Agent API: `agentSettings.systemPrompt` is a plain `string` (`agent.go`), so the system prompt always becomes a single text part with no way to attach per-part provider options. The only lever available on an `AgentCall`/`AgentStreamCall` is call-level `ProviderOptions`, which can't express "cache the first N% of the system prompt, keep the tail volatile."
### Why it matters
The classic Anthropic prompt-caching layout is a large stable system prefix (instructions, rubric, reference material) plus a small volatile suffix (recent context). Apps with that shape currently have to choose between the Agent conveniences and effective prefix caching — or drop down to the `LanguageModel` layer and build the prompt by hand.
### Suggested shape (whatever fits the API's direction)
- `agent.WithSystemPromptParts(parts ...fantasy.TextPart)` (or accept a pre-built system `Message`), letting callers attach `ProviderOptions`/part options carrying `anthropic.ProviderCacheControlOptions` — the provider plumbing already honors them; or
- a lighter `WithSystemPromptCacheBoundary(offset int)` style hint the Anthropic provider translates into a two-block system message.
Related precedent: #302 added per-message cache markers for OpenAI-compatible providers — this is the same need aimed at the system prompt through the Agent path.
Happy to take a swing at a PR if one of these directions sounds right to you.
Contributor guide
Research direction
Read agent.go to trace how agentSettings.systemPrompt becomes a system message, then inspect providers/anthropic/anthropic.go and its toPrompt handling of system TextPart options. Compare the proposed Agent API shapes with the per-message cache markers from #302. Done means the Agent path can express a cache boundary inside a system prompt while preserving the provider's existing per-part cache_control behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100