stacklok / stacklok/mecatl

Stream cumulative token usage during an active LLM response

Open
#950 0 comments 0 reactions 1 assignee View on GitHub

@jbeda is already working on this.

Since Sep 2, 2026.

Dominant language
Go
Stars
152
Forks
16
Avg merge
14h 48m
Merged PRs (30d)
536

Description

Problem\n\nToken usage is currently surfaced at the end of a completed run. During a long streamed LLM response, the operator cannot see token usage advancing.\n\n## Desired behavior\n\nUpdate and publish cumulative usage as LLM response chunks provide usage information, so clients such as mecatui can show token counts incrementing while the agent works. Final usage must remain correct and must not double-count streamed interim values plus terminal totals.\n\n## Scope to determine\n\n- Identify provider chunk usage semantics and define one provider-neutral accumulation rule.\n- Define event/protocol and client rendering changes needed for live updates.\n- Preserve accurate per-run and durable session cumulative usage across tool turns, retries, cancellation, and resume.\n- Define behavior for providers that only report terminal usage.\n\n## Acceptance idea\n\nAn offline streaming-provider test emits multiple usage-bearing chunks and proves the client-visible usage sequence advances monotonically, while the final session and result usage equal the true total exactly once.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.