agentscope-ai / agentscope-ai/agentscope
[Bug]: Media from a parallel tool result splits the tool messages, so the next request is rejected
- Langage dominant
- Python
- Étoiles
- 31.6k
- Forks
- 3.5k
- Merge moyen
- 1 j 16 h
- PR mergées (30 j)
- 103
Description
### Prerequisites
- [x] I have searched the existing issues and discussions, and this is not a duplicate.
- [x] This is a bug, not a usage question.
### Background / Description
When one assistant reply has two or more tool calls and a result that isn't the last one carries an image, audio or PDF, the OpenAI, DashScope, Moonshot and Volcengine chat formatters put the promoted media message between the tool messages:
```
assistant(tool_calls=[call_shot, call_title])
tool(call_shot)
user(image) <- inserted here
tool(call_title)
```
The Chat Completions API requires every `tool_call_id` of an assistant message to be answered by tool messages before any other message, so this request is rejected with a 400. DashScope returns the same error (see #772). Since the reply is stored in memory, every later request in the session fails too.
In v2 one assistant `Msg` holds all the tool calls and their results, and each chat formatter appends the `system-reminder` user message right after the tool message that produced the media.
### Error Messages
Not reproduced against a live endpoint. The OpenAI rule is "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'". The request shape is reproduced below.
### Steps to Reproduce
Format an `AssistantMsg` with `ToolCallBlock(call_shot)`, `ToolCallBlock(call_title)`, a `ToolResultBlock` for `call_shot` whose output has a `DataBlock` image, then a text-only `ToolResultBlock` for `call_title`:
```
OpenAIChatFormatter / DashScopeChatFormatter / MoonshotChatFormatter / VolcengineChatFormatter:
[3] assistant tool_calls=['call_shot', 'call_title']
[4] tool tool_call_id=call_shot
[5] user parts=['text', 'text', 'image_url', 'text']
[6] tool tool_call_id=call_title
```
A real case is an agent reading a screenshot and a text file in parallel.
### Environment
agentscope 2.0.8 (main at b82253b), Python 3.12, macOS.
I have a fix with tests (hold the promoted media until the run of tool messages ends) and will open a PR. This issue was written with AI assistance (Claude Code).
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Locate OpenAIChatFormatter, DashScopeChatFormatter, MoonshotChatFormatter, and VolcengineChatFormatter, then inspect their handling of media from tool results. Reproduce the two-tool-call sequence described in the issue and run the formatter tests. Done means promoted media no longer appears between tool messages, and the resulting request preserves valid tool-call ordering.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- python
- Domaine
- backend-api-design
- Type d'issue
- Bug
- Difficulté
- 3/5
- Temps estimé
- 1-2 jours
- Activité
- Active
- Clarté
- Plutôt claire
- Accessibilité débutants
- 45/100