elastic / elastic/roadmap

Token usage controls for Agent Builder

Open
#381 0 comments 0 reactions 0 assignees View on GitHub
Component: Kibana
Dominant language
No language data
Stars
6
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Following from agent traces and consumption visibility (https://github.com/elastic/roadmap/issues/218), this item covers support for enabling configurable controls on token spend and usage.

**Value proposition**
Organizations running Agent Builder at scale — across multiple teams, agents, and users — currently have no native way to cap, throttle, or allocate LLM token consumption. Native usage controls enable cost governance and scaling to a larger set of users.

**Expected outcome**
Admins can set token/spend limits that build in complexity over three phases, starting with a simple global, time-boxed budget, then a per-turn ceiling, then per-user allocation. When a limit is approached or reached, the action is configurable (warn, throttle, or block) rather than only failing mid-conversation with an opaque error. This should work for both Elastic Managed LLM connectors and self-managed/BYO-LLM connectors, including air-gapped deployments where per-user API keys are the primary enforcement point today.

**Key user stories / use cases — phased**

*Phase 1 — Global, time-based token budget *
- As an admin, I want to set an org-wide token budget for a given time window (e.g. per day/month), so I have a hard ceiling on total AI spend without needing per-agent or per-user granularity.
- As an admin, I want to be alerted (via the existing monitoring API/UI) as usage approaches the global budget, so I can react before hitting a hard limit.

*Phase 2 — Token budget per turn*
- As an agent owner, I want a per-turn/per-request token ceiling, so a single runaway conversation (e.g. excessive tool calls or context bloat) can't consume a disproportionate share of the global budget.
- As a user, I want graceful degradation (e.g. triggered summarization or a soft warning) when a turn approaches its budget, instead of a hard failure mid-task.

*Phase 3 — Token budget per user*
- As a platform/PaaS admin serving multiple internal teams through Agent Builder, I want to allocate token budgets per user or team, so no single team's usage overruns shared LLM spend.
- As an admin in a self-managed/air-gapped environment, I want per-user API keys or equivalent scoping so my existing LLM proxy's budget/throttle enforcement is respected inside Agent Builder.
- As an admin, I want per-user usage broken out in the existing monitoring views, so budget enforcement and reporting share the same data model as tracking does today.

Contributor guide

No contributing guide indexed for this repository

Research direction

No repository files, tests, or implementation entry points are identified. Start by reviewing the agent traces and consumption visibility work in roadmap issue 218, then locate the existing monitoring API/UI; done means agreeing on and implementing the phased global, per-turn, and per-user controls described here.

Written by the indexing model from the issue text.

Assessment

Domain
ai
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.