MoonshotAI / MoonshotAI/kimi-code
[feature] Expose context cache TTL selection (prompt_cache_options.ttl) for the 1h tier
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 7.5k
- Forks
- 1.2k
- Avg merge
- 11h 53m
- Merged PRs (30d)
- 350
Description
Summary
The Kimi API supports prompt_cache_options.ttl with two tiers — "5m" (default) and "1h" — for context-cache writes. Kimi Code CLI 2.0.0 has no way to select the cache tier: every request rides the default 5m, and there is no config key, env var, or CLI flag to opt into 1h.
Request: expose a provider-level option (e.g. [providers.kimi] cache_ttl = "5m" | "1h", default "5m", optionally overridable per-run via KIMI_CACHE_TTL), and pass it through as prompt_cache_options.ttl on providers/endpoints that support the field.
Motivation and cost math
Cadenced production workflows — scheduled report generation on a fixed 30-minute cadence (48 turns/day), with a long-lived supervisory session holding a stable prefix (system prompt, tool definitions, knowledge content).
Naively, the 1h tier looks pricier: cache writes cost ¥40/M vs ¥20/M for 5m (USD mirrors: $6/M vs $3/M). What makes it win is hit-renewal: after the first write, every hit within the TTL renews it at the cache-read price and no further writes are billed.
Per day, for a ~1M-token stable prefix at a 30-minute cadence:
5mdefault: the gap exceeds the TTL every time → 48 full writes/day → 48 × ¥20 = ¥960/M/day1htier: first turn writes once, the next 47 turns hit at ¥2/M and renew → ¥40 + 47 × ¥2 = ¥134/M/day
≈ 7× cheaper per day for the same workload, plus lower first-token latency on hits. Without this arithmetic, the feature request looks self-defeating ("1h costs 2× more"); the cadence case is where it pays.
Evidence
- The API field exists and is documented (
docs/api/chat, mirrored on platform.moonshot.cn / platform.kimi.ai / platform.moonshot.ai):prompt_cache_options.mode="implicit"(only supported mode; auto-writes the request prefix to cache)prompt_cache_options.ttl="5m" | "1h"— "Lifetime of the written cache. Only 5m and 1h are supported (default 5m); the two tiers are independent."- Behavior notes: cache is isolated at org granularity; a hit within the TTL renews it and is charged only the cache-read price (no re-write fee).
- Pricing pages (CNY and USD mirrors): cache writes billed per tier — 5m tier ¥20/M vs 1h tier ¥40/M (kimi-k3); cache-hit input ¥2/M (CNY) / $0.30/M (USD); regular input ¥20/M / $3/M.
- Current CLI behavior is consistent with the 5m default. Probe method (offered in full below): warm a ≥256-token prompt, then re-probe at intervals (300s / 1800s / 3600s) and compare
usage.prompt_tokens_details.cached_tokens. Intervals past the default tier's lifetime miss; turns under ~5 minutes hit. (Independent corroboration with the same method and numbers: pi-provider-kimi-code caching notes, measured TTL in [300s, 1800s).) - No knob exists in CLI 2.0.0 —
config.tomlhas no cache-related keys; the official env-vars page lists none;tui.tomloffers onlycache_expiry_hint(a reminder toggle, not a TTL control). The CLI emitsprompt_cache_key— but sendingprompt_cache_keydoes not let the caller select a cache tier; there is noprompt_cache_optionsanywhere in the client path (verified in both the open-source package and the shipped 2.0.0 binary, see below).
Proposal
- Add
cache_ttl = "5m" | "1h"under[providers.kimi](default"5m"to preserve current behavior and pricing), passed through asprompt_cache_options.ttlon providers/endpoints that document the field; omit or ignore it elsewhere for compatibility. Kimi Code talks to the coding endpoint whiledocs/api/chatdocuments the chat endpoint — if the coding endpoint does not honor this field, please document that explicitly so client code can gate on it. - Optional env override:
KIMI_CACHE_TTL=5m|1hfor per-run selection (cadenced batch jobs vs interactive sessions). - Surface the pricing implication in the option's docs (1h-tier writes cost 2× the 5m tier; hits renew the TTL and are billed at the cache-read rate), so users choose knowingly.
Code landing spots (for maintainers' convenience)
- Open-source repo:
packages/kosong/src/kosong/chat_provider/kimi.pyalready carries aprompt_cache_keyslot in the generation parameters (currently line ~98) —prompt_cache_optionswould hang naturally at the same site. - Shipped binary (2.0.0, JS bundle, agent-core-v2): cache handling lives in a
kimiOpenAITrait-styleencodeCacheKeythat only ever emits{ prompt_cache_key: key }. - These may or may not be the same codebase — could maintainers clarify which one is production for the 2.0.0 CLI?
Notes
- The
1htier pays off for cadenced workflows with stable prefixes and turn gaps between 5 and 60 minutes; it is a poor fit for continuously-changing contexts (any prefix change — includingsystem/toolschanges — cold-starts a new cache entry). - I can provide a minimal repro script for the current 5m-default behavior (warm/interval-probe with
cached_tokensevidence) on request.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in packages/kosong/src/kosong/chat_provider/kimi.py, where prompt_cache_key is already carried in generation parameters, and trace how provider configuration reaches the Kimi coding endpoint. Check whether that endpoint supports prompt_cache_options.ttl and how the shipped agent-core-v2 bundle handles encodeCacheKey. Done means a documented 5m default, selectable 1h behavior where supported, compatibility elsewhere, and coverage for the configuration or environment override.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, typescript
- Domain
- api, backend-api-design, cli
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100