Cost: the bill is 210:1 input to output — re-sending the conversation, not the work
- Dominant language
- TypeScript
- Stars
- 1
- Forks
- 0
- Avg merge
- 1h 21m
- Merged PRs (30d)
- 41
Description
## What the money shows
Across 20 recorded turns on this bench (`~/.bench/turns.json`): 56.5M input tokens, 268k output — a **210:1** ratio. A specialist re-sends its whole conversation on every tool call, so a turn's bill is (conversation size) × (re-send count) × (cache-weighted price). The work is 0.5% of the cost; the re-sent conversation is 99.5%.
The OpenRouter API confirms the scale: `usage` $58.72 of $60 credit, $37.58 today, $1.28 left. The same key serves another bench off this machine, and the API cannot attribute spend per bench — only $1.83 of the $58.72 is this bench. The bulk (~$56.89, ~$35.75 today) is the other bench, unreachable from here.
So "OpenRouter is still expensive" is not about per-token price. It is about volume: how often the conversation is re-sent, how big it is, and whether the cache discount holds.
## The levers, ranked by leverage
1. **Cache reliability — on/off switch for a 5× discount.** When caching works, 96% of input reads from cache at a fifth of fresh price. One recorded turn saved $26 by caching. The lever is the **6 of 20 turns that wrote zero cache** — they paid full price to re-send. Some may be unavoidable cold starts (first turn, after a clear, after the 5-min cache TTL lapses); some may be a real gap. The whole cost model rests on this, and it is the cheapest thing to investigate.
2. **Re-send count — the multiplier itself.** 210:1 means the conversation is re-sent roughly once per tool call. Fewer tool calls per turn = fewer re-sends. A turn that reads four files in one `Bash` call instead of four `Read` calls sends the conversation four times less. Structural, and the biggest lever, but it is how the agent works — not a setting.
3. **Conversation size — what gets re-sent.** Smaller conversation × re-send count. Context clearing and fresh tabs for separable work (the nudge in `src/shared/nudge.ts`, landed in 02a1f1a) attack this. Built, not yet proven to change behaviour on a live bench.
4. **Model price — multiplicative on volume.** A 10× cheaper model makes the same 210:1 volume cost 10× less — but a cheap model too weak to run a specialist sends work back to the expensive one, which is the "OpenRouter expensive" trap. Volume × price, in that order: volume is the bigger factor.
5. **Subscription vs cash — a fixed floor, so the lever is utilisation.** The subscription is paid regardless, so it cannot be made cheaper — but cheap roles default to OpenRouter cash when a key exists (`src/shared/role-models.ts`), even when Haiku on the already-paid plan is marginal-free. Cash spent to avoid using something already paid for. The lever is routing: prefer the subscription when its windows have capacity, spill to OpenRouter cash only when they are full.
## Ruled out as a cost lever
- **Key-splitting and the settled-cost gap** are about *seeing* the spend, not reducing it. Only 1 of this bench's 6 OpenRouter turns got a settled cost; 5 are catalogue estimates (`basis: estimated`) the code itself documents as 1.46× under. Worth fixing for honesty, does not move the bill.
- **Making the subscription cheaper** is off the table — it is a fixed floor.
## Evidence
- `~/.bench/turns.json` — 20 shapes; 56.5M input / 268k output; 96.1% cacheRead share; 6 turns with `cacheWrite: 0`.
- `openrouter.ai/api/v1/key` and `/credits`, read live: usage 58.72, total_credits 60, usage_daily 37.58, limit null.
- `~/.bench/spend.jsonl` — this bench OpenRouter $1.83 (6 turns), plan $12.59 (11 turns).
- Live catalogue: `z-ai/glm-5.2` fresh $1.19/M, cacheRead $0.221/M, out $3.74/M.
- `src/shared/role-models.ts` — cheap roles prefer OpenRouter when a key exists; `direct` (plan) fallback only with no key.
## Not verified
- Whether the 6 `cacheWrite: 0` turns are cold starts or a fixable cache gap. Cannot tell from stored shapes — `turns.json` carries no model id, so the shapes cannot be tied to OpenRouter or the plan. Needs one live run that records the model id alongside `cacheRead`/`cacheWrite`.
- Whether the other bench (the ~$35.75/day one) has the same 210:1 shape or a worse one. Not reachable from here.
- Whether the subscription windows on this machine have spare capacity — needs the OAuth usage panel, not the cost ledger.
## Proposed first step
The cheapest first step that could find real money is lever 1: run one live OpenRouter turn with the model id recorded against the cache figures, and settle whether `cache_read` comes back non-zero and whether the `cacheWrite: 0` turns correlate with cold starts. If they are cold starts, caching is already working and the lever is 2+3 (send less). If they are a real gap, that is a fix worth more than any model swap.
Contributor guide
Research direction
Start with ~/.bench/turns.json and the proposed live OpenRouter turn; inspect src/shared/role-models.ts for the routing context. Record the model id alongside cacheRead and cacheWrite, then compare the six zero-cache-write turns with cold starts. Done means establishing whether those turns indicate a cache gap or whether the cost lever is re-sending less context.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- devtools, observability
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100