agentscope-ai / agentscope-ai/agentscope

[Bug]: _compress_context_impl does not handle DataBlock before structured output

Open
#2,145 1 comment 0 reactions 1 assignee Claimed by @March-77 View on GitHub
Dominant language
Python
Stars
31.5k
Forks
3.5k
Avg merge
1d 23h
Merged PRs (30d)
95

Description

## Bug Description

`_compress_context_impl` in `_agent.py` does not sanitize or skip `DataBlock` (image/audio) before passing messages to `generate_structured_output`. This causes:

1. **Token waste**: `count_tokens` includes DataBlock tokens, but formatter discards them → compression decision based on incorrect token estimation
2. **Semantic loss**: DataBlock cannot be summarized by structured output, but occupies context window
3. **Fragility**: If using a formatter without graceful DataBlock handling, it will crash (since `context_overflow=False` causes `raise e from None`)

## Steps to Reproduce

1. Create an agent with `max_context_tokens` configured (e.g., 100000)
2. Send messages containing `DataBlock` (e.g., image via `ImageBlock`)
3. Trigger context compression (when context exceeds threshold)
4. Observe that DataBlock is passed to `generate_structured_output` without filtering

## Code Analysis

**Location**: `src/agentscope/agent/_agent.py`, `_compress_context_impl` method (lines 312-510)

**Problem flow**:
1. Line 363-369: Extract `msgs_to_compress` from `self.state.context` (raw Msg objects, may contain DataBlock)
2. Line 411-418: Directly assemble into `messages` list
3. Line 450-453: Pass directly to `self.model.generate_structured_output(messages=messages, ...)`

**No DataBlock filtering anywhere in the compression path.**

**Formatter behavior**:
| Formatter | DataBlock Handling | Crash? |
|-----------|-------------------|--------|
| `DeepSeekChatFormatter` | Falls to `else` branch (line 163), warning + skip | No |
| `OpenAIChatFormatter` | Calls `_format_openai_data_block`, formats by media_type | Depends on media_type |
| `DeepSeekMultiAgentFormatter` | `_format_agent_message` only handles TextBlock | DataBlock ignored |

## Expected Behavior

`_compress_context_impl` should sanitize `msgs_to_compress` before assembling `messages`:
- Remove DataBlock, OR
- Replace with text placeholder (e.g., `"[image attached]"`), OR
- Mark as non-compressible and skip

## Environment

- AgentScope version: latest (main branch)
- Python: 3.12
- OS: Windows

## Additional Context

This is a silent failure — DeepSeek formatter won't crash but wastes tokens and loses semantics. The fix should be in `_compress_context_impl` (lines 392-418) to filter DataBlock before passing to `generate_structured_output`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.