agentscope-ai / agentscope-ai/agentscope
[Bug]: _compress_context_impl does not handle DataBlock before structured output
- Dominant language
- Python
- Stars
- 31.5k
- Forks
- 3.5k
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 95
Description
## Bug Description
`_compress_context_impl` in `_agent.py` does not sanitize or skip `DataBlock` (image/audio) before passing messages to `generate_structured_output`. This causes:
1. **Token waste**: `count_tokens` includes DataBlock tokens, but formatter discards them → compression decision based on incorrect token estimation
2. **Semantic loss**: DataBlock cannot be summarized by structured output, but occupies context window
3. **Fragility**: If using a formatter without graceful DataBlock handling, it will crash (since `context_overflow=False` causes `raise e from None`)
## Steps to Reproduce
1. Create an agent with `max_context_tokens` configured (e.g., 100000)
2. Send messages containing `DataBlock` (e.g., image via `ImageBlock`)
3. Trigger context compression (when context exceeds threshold)
4. Observe that DataBlock is passed to `generate_structured_output` without filtering
## Code Analysis
**Location**: `src/agentscope/agent/_agent.py`, `_compress_context_impl` method (lines 312-510)
**Problem flow**:
1. Line 363-369: Extract `msgs_to_compress` from `self.state.context` (raw Msg objects, may contain DataBlock)
2. Line 411-418: Directly assemble into `messages` list
3. Line 450-453: Pass directly to `self.model.generate_structured_output(messages=messages, ...)`
**No DataBlock filtering anywhere in the compression path.**
**Formatter behavior**:
| Formatter | DataBlock Handling | Crash? |
|-----------|-------------------|--------|
| `DeepSeekChatFormatter` | Falls to `else` branch (line 163), warning + skip | No |
| `OpenAIChatFormatter` | Calls `_format_openai_data_block`, formats by media_type | Depends on media_type |
| `DeepSeekMultiAgentFormatter` | `_format_agent_message` only handles TextBlock | DataBlock ignored |
## Expected Behavior
`_compress_context_impl` should sanitize `msgs_to_compress` before assembling `messages`:
- Remove DataBlock, OR
- Replace with text placeholder (e.g., `"[image attached]"`), OR
- Mark as non-compressible and skip
## Environment
- AgentScope version: latest (main branch)
- Python: 3.12
- OS: Windows
## Additional Context
This is a silent failure — DeepSeek formatter won't crash but wastes tokens and loses semantics. The fix should be in `_compress_context_impl` (lines 392-418) to filter DataBlock before passing to `generate_structured_output`.
Contributor guide
Assessment
This issue has not been assessed yet.