apache / apache/flink-agents

[Feature] Per-agent token/cost budget guardrail (enforcement, not just reporting)

Open
#1,065 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
452
Forks
167
Avg merge
5d 5h
Merged PRs (30d)
47

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar.

### Description

#858 tracks embedding models not consistently *reporting* token usage metrics, and chat models already have a working token-accounting path that lands in the metrics layer. That's visibility. There's no enforcement layer on top of it.

What problem does it solve: a Flink Agents job is a long-running streaming application, not a one-shot script. A misbehaving action (a bad prompt loop, an unexpectedly chatty tool-call cycle, a runaway retry) can burn LLM spend continuously for as long as the job stays up, with the only signal being a metrics dashboard someone has to be watching. There's currently no built-in way to say "stop calling this model for this agent/action once N tokens (or $ cost) have been spent in a window" the way, for example, Flink itself lets you bound resource usage with slot/memory limits.

What I have in mind: a budget guardrail configured per agent (or per action/chat-model-setup), expressed as a token or cost ceiling over a sliding or fixed window, enforced on the same durable, metered chat/embedding path the token-accounting and #1062's MODEL_ROUTER work already run through — so it's a policy layered on infrastructure that already exists, not a new call path. When the ceiling is hit, the configured behavior could range from a raised error (fail the action, let normal error-handling strategy take over, same as a routing/judge failure already does in #1062) to a soft warning event for observability-only setups.

This is a runtime safety feature, distinct from #1062's routing/fallback-among-candidates concern — routing decides *which* model answers a call; this decides *whether* the call is still allowed to happen given spend so far.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

No implementation files or tests are named. Start by tracing the durable metered chat/embedding path and the token-accounting path referenced in the issue, then compare #1062's MODEL_ROUTER behavior. Done means an agreed per-agent or per-action budget policy, window and token/cost semantics, enforcement behavior, and coverage for the selected configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
ai-infra-agents, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.