agentscope-ai / agentscope-ai/agentscope-java

[Feature]: Tool-level circuit breaker — stop offering a repeatedly failing tool instead of letting the ReAct loop keep retrying it

Aperta
#2,983 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Java
Stelle
5.6k
Fork
1.3k
Merge medio
4g 12h
PR unite (30g)
77

Descrizione

## Motivation

A ReAct loop is reason → call tool → read result → reason again. When a tool returns an error the model commonly calls it again, because from the model's point of view one failure looks incidental. Nothing in the framework remembers that a tool is broken.

Against a dependency that is genuinely down — provider outage, revoked key, network partition, rate-limit wall — that costs one model call, one outbound request and seconds of latency *per retry*, and still fails. Ten consecutive failures are ten wasted reasoning rounds. There is no mechanism that says "this tool is broken right now, stop advertising it".

What exists today covers a different problem:

- `ExecutionConfig` timeout (`ToolExecutor.applyTimeout`, default 5 min) bounds one call.
- `ExecutionConfig` retry (`ToolExecutor.applyRetry`) recovers a transient blip within one call, and defaults to `maxAttempts=1` for tools (`ExecutionConfig.TOOL_DEFAULTS`). It is also currently bypassed for tool exceptions — see #2829.
- `maxIters` caps loop length, but counts iterations, not failures, so a broken tool is still called every round until the cap.
- `Toolkit.setActiveGroups` / `ToolGroup.addTool` can disable a tool, but only when some external logic decides to.

None of these carry state across calls, so none can express "this tool has failed N times in a row; leave it alone for a while". Model-side failover has an open proposal along these lines (#2863, per-candidate cooldown), but there is no equivalent for tools.

## Proposal

An opt-in, per-tool circuit breaker with the usual three states and exponential backoff:

```
CLOSED --failureThreshold consecutive failures--> OPEN
OPEN --cooldown elapsed--> HALF_OPEN (tool advertised again, as a probe)
HALF_OPEN --probe succeeds--> CLOSED
HALF_OPEN --probe fails--> OPEN (next generation, longer cooldown)
```

Two decisions are worth stating up front, because they are what make this fit an agent rather than a service mesh:

**1. Withhold the tool, do not reject the call.** A classic breaker sits between caller and dependency and fails fast once open. An agent has a strictly better option: drop the tool from the schema list sent to the model. A tool the model cannot see is a tool it cannot ask for, so the retry loop disappears at the source instead of being absorbed. No prompt has to tell the model to avoid the tool, and the model cannot argue with the decision.

Concretely this filters `ReasoningInput.tools()` per turn and leaves the `Toolkit` untouched. Mutating `ToolGroup` membership would be the other way to hide a tool, but group membership is shared mutable state: a circuit tripped while serving one session would remove the tool from every concurrent session, and it would overwrite the registrations the application declared. Per-turn filtering also means recovery needs no repair step — stop withholding and the unfiltered list is already correct.

**2. Supervision is opt-in.** Only tools named in the config are watched. Withholding a flaky weather API degrades the agent gracefully; withholding the database or filesystem tool does not degrade it, it cripples it. Requiring the supervised set to be named means a breaker can never take down a tool nobody considered.

Cooldown is `min(initialCooldown * backoffMultiplier^(generation-1), maxCooldown)`. With 60s / x2 / 600s that is 60s, 120s, 240s, 480s, 600s, 600s… A tool that keeps failing is isolated for longer, cutting both the probe traffic aimed at a struggling dependency and the tokens spent rediscovering that it is still down. The cap stops backoff from isolating a tool for hours.

Failures are classified by the typed `ToolResultState` already carried on `ToolResultEndEvent`, not by matching an error string. `ERROR` counts; `DENIED` (a permission refusal) and `INTERRUPTED` (a cancellation) do not, since neither is evidence about the dependency — counting `DENIED` would let a user who declines a confirmation prompt trip the circuit.

State is derived from a stored snapshot plus the current time, so a cooldown that elapsed while the agent was idle is recognised on the next read. No scheduler, no timer, no background thread.

## API shape (draft)

```java
ToolCircuitBreakerConfig config = ToolCircuitBreakerConfig.builder()
.monitorTools("query_weather", "query_destination_news")
.failureThreshold(3)
.initialCooldown(Duration.ofSeconds(60))
.backoffMultiplier(2.0)
.maxCooldown(Duration.ofSeconds(600))
.build();

ReActAgent agent = ReActAgent.builder()
.model(model)
.toolkit(toolkit)
.middleware(new ToolCircuitBreakerMiddleware(new ToolCircuitBreaker(config)))
.build();
```

State lives behind a `ToolCircuitBreakerStore` SPI. The in-process default needs no dependencies; a Redis-backed implementation lets replicas share one view of a broken tool, so an N-replica deployment does not send N times the failing traffic before the tool is withheld everywhere.

## Compatibility

Purely additive and inert unless the middleware is registered: no new core dependencies, no threads, no change to `ExecutionConfig`, `Toolkit` or `ReActAgent` defaults.

## Prior art

This design is in production in a downstream travel-planning agent, where the tools involved are third-party weather and destination-news APIs. Two things learned there shaped the proposal above: the whitelist is not optional (an early version supervised everything and withheld an infrastructure tool), and string-matching the error text is fragile enough that the typed state is worth depending on instead.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.