agentscope-ai / agentscope-ai/agentscope

[Bug]: GoalPipeline ignores max_retries for invalid structured outputs

未關閉 適合新手
#2,497 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
31.6k
分支
3.5k
平均合併
1 天 16 小時
30 天內合併 PR
103

描述

## Prerequisites

- [x] I have searched the existing issues and discussions, and this is not a duplicate.
- [x] This is a bug, not a usage question.

## Background / Description

`GoalPipeline.__init__` documents `max_retries` as the maximum number of retries for the executor and verifier to generate valid structured outputs, and stores it as `self.max_retries`. However, neither structured-output retry loop reads that value.

When either agent repeatedly completes without `structured_output`, `GoalPipeline.reply_stream()` invokes it indefinitely. The verifier has another unbounded path when its stream ends without a completed final message: `if final_msg is None: continue`.

This is separate from `max_iters`: `max_iters` limits executor-verifier goal attempts after an explicit `fail` verdict, while malformed or missing structured output is retried inside a single goal attempt.

### Expected behavior

`max_retries` should bound malformed/missing structured-output retries independently for the executor and verifier. Following the existing AgentScope retry convention, `max_retries=1` should allow the initial call plus one retry. Once exhausted, the pipeline should raise a clear error instead of silently appearing to finish.

HITL and external-execution events should continue to park the pipeline without consuming the malformed-output retry budget.

### Actual behavior

With `max_retries=1`, both executor and verifier can be called more than two times and continue indefinitely if they never produce structured output.

## Error Messages

```shell
No exception is raised by GoalPipeline. A guarded reproducer stopped each loop after the fifth call:
executor: guard stopped executor; calls=5; configured_retries=1
verifier: guard stopped verifier; calls=5; configured_retries=1
```

## Steps to Reproduce

The existing `StubAgent` and `_no_output` helpers in `tests/pipeline_goal_test.py` can reproduce the behavior:

```python
executor = StubAgent("executor", [[_no_output("executor")]])
verifier = StubAgent("verifier", [[_verdict("pass")]])
pipe = GoalPipeline(executor, verifier, max_retries=1)

await self._run(pipe, self.query) # never returns
```

The same occurs on the verifier path:

```python
executor = StubAgent("executor", [[_report()]])
verifier = StubAgent("verifier", [[_no_output("verifier")]])
pipe = GoalPipeline(executor, verifier, max_retries=1)

await self._run(pipe, self.query) # never returns
```

## Proposed fix

- Enforce `max_retries + 1` total attempts separately in the executor and verifier structured-output loops.
- Treat a verifier stream with no completed final message as an invalid attempt too.
- Preserve existing HITL and `max_iters` behavior.
- Add regression tests for executor exhaustion, verifier exhaustion, and success on the last allowed attempt.

I would be happy to prepare a focused PR for this.

## Environment

- AgentScope Version: `main` at `10eaaac269101e25ad9f808353c45e305c6d7231`
- Python Version: 3.11
- OS: Windows 11

貢獻指南

開啟貢獻指南

研究方向

Read `tests/pipeline_goal_test.py` first: the given `StubAgent`, `_no_output`, and `_report` reproducer cases show both failing paths. Then inspect `GoalPipeline.reply_stream` and its executor/verifier structured-output retry loops where `final_msg is None` is handled and `max_retries` should be enforced. Add regression coverage for executor exhaustion, verifier exhaustion, and success on the last allowed attempt, then run the pipeline goal tests to confirm bounded behavior and clear failure when retries are exhausted.

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
backend
Issue 類型
缺陷
難度
2/5
預估耗時
1-3 小時
活躍度
活躍
描述清晰度
描述清楚
新手友好度
78/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。