Token usage controls for Agent Builder
- Dominant language
- No language data
- Stars
- 6
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Following from agent traces and consumption visibility (https://github.com/elastic/roadmap/issues/218), this item covers support for enabling configurable controls on token spend and usage.
**Value proposition**
Organizations running Agent Builder at scale — across multiple teams, agents, and users — currently have no native way to cap, throttle, or allocate LLM token consumption. Native usage controls enable cost governance and scaling to a larger set of users.
**Expected outcome**
Admins can set token/spend limits that build in complexity over three phases, starting with a simple global, time-boxed budget, then a per-turn ceiling, then per-user allocation. When a limit is approached or reached, the action is configurable (warn, throttle, or block) rather than only failing mid-conversation with an opaque error. This should work for both Elastic Managed LLM connectors and self-managed/BYO-LLM connectors, including air-gapped deployments where per-user API keys are the primary enforcement point today.
**Key user stories / use cases — phased**
*Phase 1 — Global, time-based token budget *
- As an admin, I want to set an org-wide token budget for a given time window (e.g. per day/month), so I have a hard ceiling on total AI spend without needing per-agent or per-user granularity.
- As an admin, I want to be alerted (via the existing monitoring API/UI) as usage approaches the global budget, so I can react before hitting a hard limit.
*Phase 2 — Token budget per turn*
- As an agent owner, I want a per-turn/per-request token ceiling, so a single runaway conversation (e.g. excessive tool calls or context bloat) can't consume a disproportionate share of the global budget.
- As a user, I want graceful degradation (e.g. triggered summarization or a soft warning) when a turn approaches its budget, instead of a hard failure mid-task.
*Phase 3 — Token budget per user*
- As a platform/PaaS admin serving multiple internal teams through Agent Builder, I want to allocate token budgets per user or team, so no single team's usage overruns shared LLM spend.
- As an admin in a self-managed/air-gapped environment, I want per-user API keys or equivalent scoping so my existing LLM proxy's budget/throttle enforcement is respected inside Agent Builder.
- As an admin, I want per-user usage broken out in the existing monitoring views, so budget enforcement and reporting share the same data model as tracking does today.
Contributor guide
No contributing guide indexed for this repository
Research direction
No repository files, tests, or implementation entry points are identified. Start by reviewing the agent traces and consumption visibility work in roadmap issue 218, then locate the existing monitoring API/UI; done means agreeing on and implementing the phased global, per-turn, and per-user controls described here.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100