agentscope-ai / agentscope-ai/agentscope

[Bug]: Media from a parallel tool result splits the tool messages, so the next request is rejected

未关闭
#2,612 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
31.6k
派生
3.5k
平均合并
1 天 16 小时
30 天内合并 PR
103

描述

### Prerequisites

- [x] I have searched the existing issues and discussions, and this is not a duplicate.
- [x] This is a bug, not a usage question.

### Background / Description

When one assistant reply has two or more tool calls and a result that isn't the last one carries an image, audio or PDF, the OpenAI, DashScope, Moonshot and Volcengine chat formatters put the promoted media message between the tool messages:

```
assistant(tool_calls=[call_shot, call_title])
tool(call_shot)
user(image) <- inserted here
tool(call_title)
```

The Chat Completions API requires every `tool_call_id` of an assistant message to be answered by tool messages before any other message, so this request is rejected with a 400. DashScope returns the same error (see #772). Since the reply is stored in memory, every later request in the session fails too.

In v2 one assistant `Msg` holds all the tool calls and their results, and each chat formatter appends the `system-reminder` user message right after the tool message that produced the media.

### Error Messages

Not reproduced against a live endpoint. The OpenAI rule is "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'". The request shape is reproduced below.

### Steps to Reproduce

Format an `AssistantMsg` with `ToolCallBlock(call_shot)`, `ToolCallBlock(call_title)`, a `ToolResultBlock` for `call_shot` whose output has a `DataBlock` image, then a text-only `ToolResultBlock` for `call_title`:

```
OpenAIChatFormatter / DashScopeChatFormatter / MoonshotChatFormatter / VolcengineChatFormatter:
[3] assistant tool_calls=['call_shot', 'call_title']
[4] tool tool_call_id=call_shot
[5] user parts=['text', 'text', 'image_url', 'text']
[6] tool tool_call_id=call_title
```

A real case is an agent reading a screenshot and a text file in parallel.

### Environment

agentscope 2.0.8 (main at b82253b), Python 3.12, macOS.

I have a fix with tests (hold the promoted media until the run of tool messages ends) and will open a PR. This issue was written with AI assistance (Claude Code).

贡献指南

打开贡献指南

调研方向

Locate OpenAIChatFormatter, DashScopeChatFormatter, MoonshotChatFormatter, and VolcengineChatFormatter, then inspect their handling of media from tool results. Reproduce the two-tool-call sequence described in the issue and run the formatter tests. Done means promoted media no longer appears between tool messages, and the resulting request preserves valid tool-call ordering.

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
backend-api-design
Issue 类型
缺陷
难度
3/5
预计耗时
1-2 天
活跃度
活跃
描述清晰度
基本清楚
新手友好度
45/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。