agentscope-ai / agentscope-ai/agentscope

[Bug]: interrupting a streaming tool call persists truncated JSON and breaks subsequent turns

未关闭
#2,469 8 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
31.5k
派生
3.5k
平均合并
1 天 23 小时
30 天内合并 PR
95

描述

### Prerequisites

- [x] I have searched the existing [issues](https://github.com/agentscope-ai/agentscope/issues) and [discussions](https://github.com/agentscope-ai/agentscope/discussions), and this is not a duplicate.
- [x] This is a bug, not a usage question. (For questions, please use [Discussions](https://github.com/agentscope-ai/agentscope/discussions/new?category=general) instead.)

### Background / Description

When a model stream is interrupted while it is still generating tool-call arguments, AgentScope can persist the partially accumulated `ToolCallBlock.input` into the conversation history.

If the interruption occurs before the tool-call arguments form valid JSON, the malformed input is still retained in the session history. AgentScope later marks the tool call as `finished` and appends an interrupted `ToolResultBlock`, but the original malformed `ToolCallBlock.input` remains unchanged.

On the next user turn, `OpenAIChatFormatter` replays this malformed value directly as:

```text
tool_calls[].function.arguments
```

For OpenAI-compatible backends that validate historical tool calls, such as vLLM with the `qwen3_coder` tool-call parser, the whole request is then rejected with HTTP 400.

Because the malformed tool call remains in persistent session history, every subsequent turn in the same session fails. Creating a new session works normally.

This is not a context-length issue. In the observed session, only about 34k of a 128k context window was used.

### Observed persisted state

The interrupted tool call was stored approximately as:

```json
{
"type": "tool_call",
"id": "chatcmpl-tool-...",
"name": "Write",
"input": "{\"file_path\": \"/workspace/scratch/verify2896.py\", \"content\": \"Verify the standard DP ... print(f\"MISMATCH: s1={s1} s2={'",
"state": "finished"
}
```

The corresponding result was:

```text
The tool call has been interrupted by the user.
```

The `input` field is clearly incomplete JSON: the `content` string and outer object are never closed.

### Suspected root cause

The current interruption path appears to be:

1. `src/agentscope/model/_base.py`
- catches `asyncio.CancelledError`
- keeps the accumulated partial streaming response
- returns it as an interrupted `ChatResponse`

2. `src/agentscope/model/_utils.py`
- `_StreamAccumulator` concatenates partial `ToolCallBlock.input` fragments
- no JSON-validity check is performed before `build()`

3. `src/agentscope/agent/_agent.py`
- `_reasoning_impl()` saves the completed/interrupted response content into context
- interruption cleanup later marks the tool call as `finished`
- an interrupted tool result is appended
- the malformed `ToolCallBlock.input` itself is not removed or sanitized

4. `src/agentscope/formatter/_openai_formatter.py`
- historical `ToolCallBlock.input` is replayed unchanged as `function.arguments`

So the lifecycle becomes:

```text
streaming tool arguments

user interrupts

partial ToolCallBlock is preserved

malformed JSON is saved to context

tool call is marked finished

next turn replays malformed arguments

OpenAI-compatible backend returns HTTP 400
```

### Expected behavior

After an interruption, the session history should remain valid and the user should be able to continue the same conversation.

An incomplete tool call should not later be replayed as a normal historical tool call containing malformed JSON.

Possible implementations could discard, sanitize, or otherwise make interrupted incomplete tool-call arguments safe for history replay.

### Related work

This seems related to, but not fully covered by:

- #1806 / #1995 — graceful agent interruption and context/tool-state consistency
- #2012 — repair truncated tool-call input in Anthropic/Gemini/Ollama formatters
- #2075 — Web UI handling of incomplete JSON during tool-call streaming/interruption

In particular, #2012 already notes that truncated `ToolCallBlock.input` can be produced by streaming interruption, but `OpenAIChatFormatter` still forwards the raw string. That is safe for the OpenAI SDK itself, but not necessarily for strict OpenAI-compatible backends that parse historical tool-call arguments.

### Error Messages

```shell
openai.BadRequestError: Error code: 400 - {
'error': {
'message': 'Unterminated string starting at: line 1 column 62 (char 61)',
'type': 'BadRequestError',
'param': None,
'code': 400
}
}

Model Qwen3.8-27B exhausted all 1 attempt(s)

Reply failed; reported to the client as invalid_request.
```

### Steps to Reproduce

1. Start an AgentScope application with:

- streaming enabled
- an OpenAI-compatible model backend
- a tool whose arguments may contain a long string, e.g. `Write(file_path, content)`
- persistent session storage

2. Ask the agent to generate and write a sufficiently long source file so that the `content` argument is emitted over multiple streaming chunks.

3. While the model is still streaming the `Write` tool-call arguments, click the Web UI **Stop / Interrupt** button.

4. Inspect the persisted conversation context.

An interrupted tool call can be found whose `input` contains incomplete JSON, for example:

```json
{
"name": "Write",
"input": "{\"file_path\":\"/workspace/test.py\",\"content\":\"...unfinished",
"state": "finished"
}
```

An interrupted tool result is also present.

5. Send any new user message in the same session.

6. Observe that AgentScope replays the malformed historical tool-call arguments to the OpenAI-compatible backend.

7. With vLLM using the `qwen3_coder` tool-call parser, the request fails with:

```text
Unterminated string starting at ...
```

and HTTP 400 is returned.

8. Send another message in the same session.

The same error occurs again because the malformed historical tool call is still present.

9. Create a new session and send a message.

The request succeeds because the malformed historical tool call is no longer part of the context.

### Environment

- AgentScope Version: 2.0.7
- Python Version: 3.11.15
- OS: Linux / Docker
- Model: Qwen3.8-27B
- Model Backend: vLLM 0.27.1
- API: OpenAI-compatible Chat Completions
- Context Size: 128k
- Tool Call Parser: qwen3_coder
- Streaming: enabled
- Session Storage: Redis

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。