microsoft / microsoft/amplifier

[provider-anthropic] Support 1h prompt-cache TTL — verified working against the live API, no beta header required

Open
#337 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.1k
Forks
261
Avg merge
3h 28m
Merged PRs (30d)
13

Description

Summary

provider-anthropic hardcodes cache_control: {"type": "ephemeral"} (5-minute TTL) at all three breakpoint sites (__init__.py:2136, :3719, :3745, verified on main 2026-07-20). There is no config to request the 1-hour TTL.

For human-paced interactive sessions, any gap >5 minutes between turns expires the cache and the next call re-writes the FULL context at cache-write rates. Observed: a single 282k-token cache-write event after one thinking pause (session data in the companion loop-streaming issue). With large system prompts (bundles + MCP tool schemas commonly reach 130k+ tokens base), these expiry re-writes dominate cost.

Verified against the live API (2026-07-20)

ttl: "1h" is accepted today with no beta header:

  • cache_control: {type: ephemeral, ttl: "1h"} -> HTTP 200
  • usage returns a per-TTL breakdown: cache_creation: {ephemeral_5m_input_tokens: 0, ephemeral_1h_input_tokens: 4419}

This also means the comment in _cost.py:131 ("Anthropic's usage object returns a single cache_creation_input_tokens count") is stale — the API now distinguishes tiers, so 1h writes (2x base input) can be cost-tracked correctly.

Suggested change

  1. New provider config key cache_ttl: "5m" | "1h" (default "5m", preserving current behavior), applied at all three breakpoint sites.
  2. _cost.py: read the cache_creation per-TTL breakdown and price 1h writes at 2x base input.

Economics: 1h writes cost 2x vs 1.25x, but for sessions with several >5min gaps per hour, converting full-context re-writes (1.25x each) into cache reads (0.1x) is a large net win.

Happy to submit the PR — small, config-gated, default-off.

Related

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the three cache_control breakpoint sites in init.py at lines 2136, 3719, and 3745, then read _cost.py:131 and the provider configuration handling. Done means a default 5m setting preserves current behavior, 1h requests are applied at every breakpoint, and the per-TTL cache_creation usage is priced correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.