aws / aws/amazon-q-developer-cli

Telemetry for context window usage over time

Open
#2,832 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2k
Forks
439
PR merge metrics
No merged PRs in 30d

Description

Goal - have visibility into context window usage to help identify when and why a user might run out of context window and need to compact or start a new session.

The current /usage command does a great job of describing what areas are specifically consuming context.

Current context window (60630 of 200k tokens used)
|██████████████|██████████████████████████████████████████████████████████████████ 30.32%

█ Context files: ~120 tokens (0.06%)
█ Tools: ~35490 tokens (17.75%)
█ Q responses: ~2120 tokens (1.06%)
█ Your prompts: ~22910 tokens (11.45%)

Ideally we'd have logging or other instrumentation that gives this information at meaningful events:

1. Session Startup
2. Context overflow
3. After compacting
4. If it is easy to have this information recorded during the session as well, this could also be useful.

If we can log the number of "turns" or other increments it would help us see if tokens get consumed by large single actions, or a collection of actions.

When this is instrumented and segmented we can start to characterize the "common" experiences that people have.

Things like - are there patterns for people who run out of the context window more? Is it b/c tool responses are chewing up context? Or MCP tool descriptions consume too much? Or users have too many context files?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.