agentscope-ai / agentscope-ai/agentscope

[Bug]: GoalPipeline ignores max_retries for invalid structured outputs

Offen
#2,497 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
31.5k
Forks
3.5k
Ø Merge
1 T. 23 Std.
Gemergte PRs (30 T.)
95

Beschreibung

## Prerequisites

- [x] I have searched the existing issues and discussions, and this is not a duplicate.
- [x] This is a bug, not a usage question.

## Background / Description

`GoalPipeline.__init__` documents `max_retries` as the maximum number of retries for the executor and verifier to generate valid structured outputs, and stores it as `self.max_retries`. However, neither structured-output retry loop reads that value.

When either agent repeatedly completes without `structured_output`, `GoalPipeline.reply_stream()` invokes it indefinitely. The verifier has another unbounded path when its stream ends without a completed final message: `if final_msg is None: continue`.

This is separate from `max_iters`: `max_iters` limits executor-verifier goal attempts after an explicit `fail` verdict, while malformed or missing structured output is retried inside a single goal attempt.

### Expected behavior

`max_retries` should bound malformed/missing structured-output retries independently for the executor and verifier. Following the existing AgentScope retry convention, `max_retries=1` should allow the initial call plus one retry. Once exhausted, the pipeline should raise a clear error instead of silently appearing to finish.

HITL and external-execution events should continue to park the pipeline without consuming the malformed-output retry budget.

### Actual behavior

With `max_retries=1`, both executor and verifier can be called more than two times and continue indefinitely if they never produce structured output.

## Error Messages

```shell
No exception is raised by GoalPipeline. A guarded reproducer stopped each loop after the fifth call:
executor: guard stopped executor; calls=5; configured_retries=1
verifier: guard stopped verifier; calls=5; configured_retries=1
```

## Steps to Reproduce

The existing `StubAgent` and `_no_output` helpers in `tests/pipeline_goal_test.py` can reproduce the behavior:

```python
executor = StubAgent("executor", [[_no_output("executor")]])
verifier = StubAgent("verifier", [[_verdict("pass")]])
pipe = GoalPipeline(executor, verifier, max_retries=1)

await self._run(pipe, self.query) # never returns
```

The same occurs on the verifier path:

```python
executor = StubAgent("executor", [[_report()]])
verifier = StubAgent("verifier", [[_no_output("verifier")]])
pipe = GoalPipeline(executor, verifier, max_retries=1)

await self._run(pipe, self.query) # never returns
```

## Proposed fix

- Enforce `max_retries + 1` total attempts separately in the executor and verifier structured-output loops.
- Treat a verifier stream with no completed final message as an invalid attempt too.
- Preserve existing HITL and `max_iters` behavior.
- Add regression tests for executor exhaustion, verifier exhaustion, and success on the last allowed attempt.

I would be happy to prepare a focused PR for this.

## Environment

- AgentScope Version: `main` at `10eaaac269101e25ad9f808353c45e305c6d7231`
- Python Version: 3.11
- OS: Windows 11

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.