agentscope-ai / agentscope-ai/agentscope-java

[Bug]: ExecutionConfig retry settings are ignored for tool failures

Abierto
#2,829 1 comentario 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
Java
Estrellas
5.6k
Forks
1.3k
Merge medio
4 d 12 h
PR fusionados (30 d)
77

Descripción

**Describe the bug**

When a tool call fails with an exception, the retry policy configured in `ExecutionConfig` (`maxAttempts`, `retryOn`, backoff) is never applied. The tool is invoked exactly once and the failure is immediately returned as a `ToolResultBlock` error result, even for clearly retryable errors such as `IOException` (network/transport failures).

Root cause: `ToolExecutor.executeWithInfrastructure(...)` runs the compatibility entry `execute(...)`, which converts tool failures into `ToolResultBlock` error results inside `executeCore(...)` **before** the scheduling/timeout/retry layers run. By the time `applyRetry(...)` executes, failures are already completed results, so `retryWhen` never sees an error signal. Only the error emitted by the `timeout(...)` operator can still reach the retry layer.

**To Reproduce**

Steps to reproduce the behavior:

1. Add the following test class to `agentscope-core/src/test/java/io/agentscope/core/tool/RetryReproTest.java`:

```java
package io.agentscope.core.tool;

import io.agentscope.core.message.ToolResultBlock;
import io.agentscope.core.message.ToolUseBlock;
import io.agentscope.core.model.ExecutionConfig;
import java.io.IOException;
import java.time.Duration;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
import org.junit.jupiter.api.Test;
import reactor.core.publisher.Mono;

/** Minimal reproduction for tool retry behavior. */
class RetryReproTest {

static final AtomicInteger CALLS = new AtomicInteger(0);

static class FlakyTools {
@Tool(name = "flaky", description = "Fails once, then recovers")
public String flaky() throws IOException {
if (CALLS.incrementAndGet() == 1) {
throw new IOException("Network down");
}
return "recovered";
}
}

@Test
void reproduce() {
Toolkit toolkit = new Toolkit();
toolkit.registerTool(new FlakyTools());

ExecutionConfig config =
ExecutionConfig.builder()
.maxAttempts(3) // 1 initial attempt + up to 2 retries
.initialBackoff(Duration.ofMillis(100))
.maxBackoff(Duration.ofMillis(200))
.retryOn(ExecutionConfig.RETRYABLE_ERRORS) // IOException is retryable
.build();

ToolUseBlock call =
ToolUseBlock.builder()
.id("call-1")
.name("flaky")
.input(Map.of())
.content("{}") // required: schema validation reads ToolUseBlock.content()
.build();

List results =
toolkit.callTools(List.of(call), config, null, null).block();

System.out.println("REPRO attempts = " + CALLS.get());
System.out.println("REPRO state = " + results.get(0).getState());
System.out.println("REPRO output = " + results.get(0).getOutput());
}

@Test
void timeoutComparison() {
AtomicInteger timeoutCalls = new AtomicInteger(0);
Toolkit toolkit = new Toolkit();
toolkit.registerTool(
new AgentTool() {
@Override
public String getName() {
return "never";
}

@Override
public String getDescription() {
return "never completes";
}

@Override
public Map getParameters() {
return Map.of();
}

@Override
public Mono callAsync(ToolCallParam param) {
timeoutCalls.incrementAndGet();
return Mono.never();
}
});

ExecutionConfig config =
ExecutionConfig.builder()
.timeout(Duration.ofMillis(100))
.maxAttempts(3)
.initialBackoff(Duration.ofMillis(50))
.maxBackoff(Duration.ofMillis(100))
.build();

ToolUseBlock call =
ToolUseBlock.builder()
.id("call-2")
.name("never")
.input(Map.of())
.content("{}")
.build();

List results =
toolkit.callTools(List.of(call), config, null, null).block();

System.out.println("REPRO timeout attempts = " + timeoutCalls.get());
System.out.println("REPRO timeout state = " + results.get(0).getState());
System.out.println("REPRO timeout output = " + results.get(0).getOutput());
}
}
```

2. Run it from the repository root:

```
mvn -pl agentscope-core test -Dtest=RetryReproTest -Dsurefire.failIfNoSpecifiedTests=false
```

3. See the printed output and the surefire report (see **Error messages**).

**Expected behavior**

The first failure should surface as an error signal to the retry layer, so the tool is re-invoked and recovers:

```
REPRO attempts = 2
REPRO state = RUNNING
REPRO output = ["recovered"]
```

(`RUNNING` is the framework's default state for a plain successful tool result.) In the timeout comparison, the tool should be re-invoked on every attempt and the final error result should keep the root cause:

```
REPRO timeout attempts = 3
REPRO timeout state = ERROR
REPRO timeout output = [Error: Tool execution failed: Tool execution timeout after PT0.1S]
```

**Error messages**

No exception is thrown; the symptom is the missing retry. Actual output on the current `main`:

```
REPRO attempts = 1
REPRO state = ERROR
REPRO output = [Error: Tool execution failed: Network down]
REPRO timeout attempts = 1
REPRO timeout state = ERROR
REPRO timeout output = [Error: Tool execution failed: Retries exhausted: 2/2]
```

Observations:

- For the flaky tool, no `Retrying tool call 'flaky' ...` warning ever appears: the retry layer never runs for tool exceptions.
- The timeout comparison isolates the two halves of the bug: the timeout error signal **is** retried (the `Retries exhausted: 2/2` message proves 3 attempts), but the tool is still only invoked **once** (`REPRO timeout attempts = 1`) — retries merely re-subscribe the already-assembled chain instead of re-invoking the tool. Additionally, the exhausted-retry message replaces the real cause (`Tool execution timeout after PT0.1S`) with the generic `Retries exhausted: 2/2`.

**Environment (please complete the following information):**

- AgentScope-Java Version: 2.0.3-SNAPSHOT (reproduced on `main` at commit `ddad42e3`; the same code path exists in earlier 2.0.x)
- Java Version: 17
- OS: macOS (the issue is OS-independent)

**Additional context**

- Relevant code: `ToolExecutor.executeCore(...)` converts failures with `ToolResultBlock.error(...)`; `ToolExecutor.executeWithInfrastructure(...)` then pipes that chain through `applyScheduling` / `applyTimeout` / `applyRetry` / `applyShutdownGuard`, so `applyRetry` only ever sees completed results (except timeouts).
- A fix needs the tool invocation to keep failures as reactive error signals until after the timeout/retry layers (for example, a separate execution channel that is re-invoked per attempt), while preserving the existing `Toolkit.callTool(...)` contract that direct calls still receive error results.
- The fix must also keep `ToolSuspendException` (external-execution suspension) from ever being retried, even when it is wrapped in reflection/future/arbitrary exceptions, since replaying a suspension duplicates side effects.
- I am preparing a fix and will open a PR referencing this issue.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.