Stream cumulative token usage during an active LLM response
Open
@jbeda is already working on this.
Since Sep 2, 2026.
- Dominant language
- Go
- Stars
- 152
- Forks
- 16
- Avg merge
- 14h 48m
- Merged PRs (30d)
- 536
Description
Problem\n\nToken usage is currently surfaced at the end of a completed run. During a long streamed LLM response, the operator cannot see token usage advancing.\n\n## Desired behavior\n\nUpdate and publish cumulative usage as LLM response chunks provide usage information, so clients such as mecatui can show token counts incrementing while the agent works. Final usage must remain correct and must not double-count streamed interim values plus terminal totals.\n\n## Scope to determine\n\n- Identify provider chunk usage semantics and define one provider-neutral accumulation rule.\n- Define event/protocol and client rendering changes needed for live updates.\n- Preserve accurate per-run and durable session cumulative usage across tool turns, retries, cancellation, and resume.\n- Define behavior for providers that only report terminal usage.\n\n## Acceptance idea\n\nAn offline streaming-provider test emits multiple usage-bearing chunks and proves the client-visible usage sequence advances monotonically, while the final session and result usage equal the true total exactly once.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.