Bug: Copilot CLI 1.0.82 BYOK silently disables prompt caching (~5x cost)
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 11.2k
- Forks
- 1.9k
- Avg merge
- 14h 16m
- Merged PRs (30d)
- 6
Description
Summary
Copilot CLI 1.0.82 in BYOK mode sends chat requests that carry no prompt-cache declaration, so prompt caching is effectively disabled for the whole session. Every turn re-sends the entire growing context at full price — provider usage shows cached_tokens=0 and cache_write/cache_creation_tokens=0 on every request across the session, even though the prefix is stable.
Net effect: ~5x cost for normal multi-turn BYOK sessions.
Setup
- Copilot CLI 1.0.82, BYOK mode (custom OpenAI-compatible
/chat/completionsendpoint serving an Anthropic model,claude-sonnet-5) - Compare baseline: same machine, same endpoint/model, 1.0.80 BYOK → prompt caching works (~97% hit)
- Also tested 1.0.82 in GitHub-subscription mode on the same machine → caching works, so the build is cache-capable
Expected
Once a conversation prefix stabilizes, the provider should report cached_tokens > 0 and small cache_creation_tokens for the new tail (that's what 1.0.80 BYOK and subscription mode report).
Actual
Every request reports zero cache metrics:
"prompt_tokens_details": {
"text_tokens": 146022,
"cached_tokens": 0,
"cache_write_tokens": 0,
"cache_creation_tokens": 0,
"cache_creation_token_details": { "ephemeral_5m_input_tokens": 0 }
},
"cache_read_input_tokens": 0
Over a 150-request session, text_tokens climbed monotonically 118,966 → 285,085 while cached_tokens and cache_write/creation_tokens stayed 0 every time. The zero cache-write/creation is the strong signal: even a cold miss should write the prefix so the next turn can hit; a persistent 0 write means the request never asks for a cache at all.
Evidence
| Metric | 1.0.82 BYOK (broken) | 1.0.80 BYOK (baseline) |
|---|---|---|
| Requests | 150 | 320 |
| Input tokens | 30,075,042 | 100,762,401 |
| Cached tokens | 0 | 98,424,636 |
| Cache hit rate | 0% | 97.7% |
| Cost | $61.21 | $28.92 |
With normal caching, 30.1M input ≈ $11.4; at 0% it costs ≈ $60.2 → ~$49 (~5x) wasted in a single session.
Root-cause hypothesis
The 1.0.82 BYOK request builder omits the prompt-cache declaration (cache_control / ephemeral-cache policy) that subscription mode injects. Because BYOK calls the user's endpoint directly, the missing declaration is only visible to BYOK users.
Severity
High — silent runaway cost on BYOK setups; no CLI error, only a much larger bill. Requires monitoring provider cache hit-rate to detect.
Suggested fix
Make the BYOK path send the same prompt-cache declaration used by subscription mode.
Repro note
I cannot reproduce right now (downgraded to 1.0.80 where caching works). Reproducing requires running 1.0.82 in BYOK against an Anthropic-model endpoint and reading provider cache accounting. Happy to help test a newer build backported if there's a fix candidate.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the BYOK request builder and compare its request construction with the subscription path and the 1.0.80 behavior described in the issue. Inspect whether the prompt-cache declaration is present, then exercise a multi-turn BYOK session against an Anthropic-model endpoint and verify that cache-write and cache-read metrics are no longer zero.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- shell
- Domain
- api, cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100