kestra-io / kestra-io/plugin-ai
No cap on the cumulative token spend of a run
- Dominant language
- Java
- Stars
- 9
- Forks
- 24
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 15
Description
The number of tokens the model can generate in one response is limited by `configuration.maxToken`. There is no limit on how many tokens are consumed by a run overall. With tools, `maxSequentialToolsInvocations` is unlimited by default, so an unbounded number of model calls can be made in one execution, and the context is extended with every tool result. The cost of a run is only visible in the Outputs tab once it has finished.
Counting the tokens before the first call was considered and left out: the prompt is a template resolved at runtime, so no count can be shown before a run is started, and an exact count is model-specific with no local tokenizer available for most providers. Neither problem applies to the token usage returned with each response, which is already aggregated across the whole tool loop.
## How to reproduce
With a `hello-world` flow in `company.team`:
```yaml
id: token_budget
namespace: company.team
tasks:
- id: agent
type: io.kestra.plugin.ai.agent.AIAgent
prompt: Execute the hello-world flow in company.team ten times, one call at a time, and report each result.
provider:
type: io.kestra.plugin.ai.provider.GoogleGemini
modelName: gemini-3.5-flash
apiKey: "{{ secret('GEMINI_API_KEY') }}"
tools:
- type: io.kestra.plugin.ai.tool.KestraFlow
namespace: company.team
flowId: hello-world
description: Say hello
configuration:
maxToken: 10
```
Ten model calls are made in one execution, and the context is extended by each tool result. In the Outputs tab, `tokenUsage.inputTokenCount` is the sum across all ten calls. Only the length of each response was limited by `maxToken: 10`. No property exists that would have stopped the run at a chosen number of tokens.
Reported from a daily commit digest: 43,825 input tokens for one day of activity when a raw API response was passed into the prompt, against 8,751 for the same answer once the input was reduced first. With a backfill, one click can be thirty such executions.
## To do
- A cumulative token budget should be offered on the tasks that call a model, checked against the token usage returned with each response, with the run stopped and failed once the budget is exceeded.
- The unlimited default on `maxSequentialToolsInvocations` should be reconsidered, since the number of model calls in an agent run is what the cost depends on most.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the AI task execution and the existing maxToken, maxSequentialToolsInvocations, and tokenUsage handling described in the issue. Determine where usage from each response is aggregated, then define the budget check and failure behavior; done means a cumulative limit stops an over-budget run and the unlimited tool-invocation default is addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- ai, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100