feat(budget): use weighted token-cost accounting for token ceilings
@JAORMX is already working on this.
Since Sep 7, 2026.
- Dominant language
- Go
- Stars
- 152
- Forks
- 16
- Avg merge
- 14h 48m
- Merged PRs (30d)
- 536
Description
Problem
The current MaxRunTokens / MaxTeamTokens budget brake uses raw cumulative input + output token count. That is intentionally simple, but it is a poor proxy for provider cost: input tokens are typically cheaper than output tokens, and cache-read input tokens are cheaper again. A long cached prompt can therefore trip the brake much earlier than its cost justifies.
Desired change
Introduce a provider-neutral weighted token-cost accounting value for token-budget enforcement and reporting. Use integer fixed-point accounting (for example, millitokens) rather than floating point, so aggregation, persistence, comparisons, and tests are deterministic.
The relative cost multipliers must be operator-configurable at build/configuration time. Initial defaults, expressed relative to output tokens, should be:
- output tokens:
1.00 - non-cache input tokens:
0.20 - cache-read input tokens:
0.02 - cache-write input tokens: explicitly configured, not silently assumed to have the same price as either ordinary input or cache reads
Cache-write accounting gotcha
CacheReadTokens and CacheWriteTokens are usage facets, not independent extra token totals. Both are reported as subsets of InputTokens after provider normalization. The accounting model must partition the inclusive input total without double-counting cache reads or cache writes. In particular, a footer value such as ↑7.7M ⊕794.8K does not mean 8.5M input tokens.
The implementation must define and test how a provider report that has cache-read and cache-write input tokens is partitioned into ordinary input, cache-read input, and cache-write input. It must also handle missing or malformed provider cache-write information conservatively and document the chosen policy.
Scope and design questions
- Preserve raw
session.Usagetoken counters for observability and provider-neutral usage reporting. Add a distinct derived weighted-cost value rather than redefiningUsage.TotalTokens(). - Give cache-write input its own configurable multiplier because its provider pricing may differ from both ordinary input and cache-read input.
- Decide the public/operator naming carefully. A budget named
max-run-tokenscannot silently change units without an explicit compatibility and migration story; consider an explicitly cost-oriented budget/config surface. - Make multipliers operator-tier configuration, with validation, documented defaults, precedence, and an explicit behavior for absent configuration.
Acceptance criteria
- Per-run and team-wide budget gates use a deterministic fixed-point weighted cost.
- Operator configuration can set validated multipliers for output, ordinary input, cache-read input, and cache-write input; documented defaults cover the 1.00 / 0.20 / 0.02 initial values.
- Input, output, cache-read, and cache-write accounting is correct across supported providers. Cache read/write facets are not double-counted against inclusive
InputTokens. - Raw token usage remains available and semantically unchanged.
- The run and team budget boundary guarantees remain unchanged: in-flight work finishes; the run/team stops at the next safe boundary.
- Persisted sessions and resumed runs retain enough accounting state for the cumulative budget to remain correct after restart.
- Wire/UI/docs make the unit and the raw-token-versus-cost distinction unambiguous.
- Existing API compatibility is evaluated; if an engine exported API changes, update
engine/apiandengine/CHANGELOG.mdas required.
Context
Current budget logic is session.Usage.TotalTokens() in engine/agent/loop.go (budgetExhausted) and the team accumulator in engine/agent/teamsupervisor.go. Usage normalization makes cache reads and writes subsets of input; see docs/design/IMPLEMENTATION-NOTES.md under “Token budget” and the provider usage-normalization section.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.