microsoft / microsoft/amplifier
[provider-anthropic] Support 1h prompt-cache TTL — verified working against the live API, no beta header required
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.1k
- Forks
- 261
- Avg merge
- 3h 28m
- Merged PRs (30d)
- 13
Description
Summary
provider-anthropic hardcodes cache_control: {"type": "ephemeral"} (5-minute TTL) at all three breakpoint sites (__init__.py:2136, :3719, :3745, verified on main 2026-07-20). There is no config to request the 1-hour TTL.
For human-paced interactive sessions, any gap >5 minutes between turns expires the cache and the next call re-writes the FULL context at cache-write rates. Observed: a single 282k-token cache-write event after one thinking pause (session data in the companion loop-streaming issue). With large system prompts (bundles + MCP tool schemas commonly reach 130k+ tokens base), these expiry re-writes dominate cost.
Verified against the live API (2026-07-20)
ttl: "1h" is accepted today with no beta header:
cache_control: {type: ephemeral, ttl: "1h"}-> HTTP 200- usage returns a per-TTL breakdown:
cache_creation: {ephemeral_5m_input_tokens: 0, ephemeral_1h_input_tokens: 4419}
This also means the comment in _cost.py:131 ("Anthropic's usage object returns a single cache_creation_input_tokens count") is stale — the API now distinguishes tiers, so 1h writes (2x base input) can be cost-tracked correctly.
Suggested change
- New provider config key
cache_ttl: "5m" | "1h"(default "5m", preserving current behavior), applied at all three breakpoint sites. _cost.py: read thecache_creationper-TTL breakdown and price 1h writes at 2x base input.
Economics: 1h writes cost 2x vs 1.25x, but for sessions with several >5min gaps per hour, converting full-context re-writes (1.25x each) into cache reads (0.1x) is a large net win.
Happy to submit the PR — small, config-gated, default-off.
Related
- Companion issue: https://github.com/microsoft/amplifier/issues/336
- Existing PRs #58 / #65 on amplifier-module-provider-anthropic cover missing Fable rates in
_RATES; the per-TTL cost handling here composes with those.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the three cache_control breakpoint sites in init.py at lines 2136, 3719, and 3745, then read _cost.py:131 and the provider configuration handling. Done means a default 5m setting preserves current behavior, 1h requests are applied at every breakpoint, and the per-TTL cache_creation usage is priced correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100