agentscope-ai / agentscope-ai/agentscope

[Bug]: _compress_context_impl does not handle DataBlock before structured output

未關閉
#2,145 1 則留言 0 個 reaction 已指派 1 人 已被 @March-77 認領 在 GitHub 檢視
主要語言
Python
星號
31.6k
分支
3.5k
平均合併
1 天 16 小時
30 天內合併 PR
103

描述

## Bug Description

`_compress_context_impl` in `_agent.py` does not sanitize or skip `DataBlock` (image/audio) before passing messages to `generate_structured_output`. This causes:

1. **Token waste**: `count_tokens` includes DataBlock tokens, but formatter discards them → compression decision based on incorrect token estimation
2. **Semantic loss**: DataBlock cannot be summarized by structured output, but occupies context window
3. **Fragility**: If using a formatter without graceful DataBlock handling, it will crash (since `context_overflow=False` causes `raise e from None`)

## Steps to Reproduce

1. Create an agent with `max_context_tokens` configured (e.g., 100000)
2. Send messages containing `DataBlock` (e.g., image via `ImageBlock`)
3. Trigger context compression (when context exceeds threshold)
4. Observe that DataBlock is passed to `generate_structured_output` without filtering

## Code Analysis

**Location**: `src/agentscope/agent/_agent.py`, `_compress_context_impl` method (lines 312-510)

**Problem flow**:
1. Line 363-369: Extract `msgs_to_compress` from `self.state.context` (raw Msg objects, may contain DataBlock)
2. Line 411-418: Directly assemble into `messages` list
3. Line 450-453: Pass directly to `self.model.generate_structured_output(messages=messages, ...)`

**No DataBlock filtering anywhere in the compression path.**

**Formatter behavior**:
| Formatter | DataBlock Handling | Crash? |
|-----------|-------------------|--------|
| `DeepSeekChatFormatter` | Falls to `else` branch (line 163), warning + skip | No |
| `OpenAIChatFormatter` | Calls `_format_openai_data_block`, formats by media_type | Depends on media_type |
| `DeepSeekMultiAgentFormatter` | `_format_agent_message` only handles TextBlock | DataBlock ignored |

## Expected Behavior

`_compress_context_impl` should sanitize `msgs_to_compress` before assembling `messages`:
- Remove DataBlock, OR
- Replace with text placeholder (e.g., `"[image attached]"`), OR
- Mark as non-compressible and skip

## Environment

- AgentScope version: latest (main branch)
- Python: 3.12
- OS: Windows

## Additional Context

This is a silent failure — DeepSeek formatter won't crash but wastes tokens and loses semantics. The fix should be in `_compress_context_impl` (lines 392-418) to filter DataBlock before passing to `generate_structured_output`.

貢獻指南

開啟貢獻指南

研究方向

Start in `src/agentscope/agent/_agent.py` at `_compress_context_impl` (lines 312-510), focusing on how `msgs_to_compress` is built (around 363-369) and converted into `messages` (411-418) before `self.model.generate_structured_output(...)` at 450-453. Confirm where `DataBlock` objects remain in the flow and add filtering/sanitization at that point so structured output only receives safe message content. Reproduce with a context that includes `ImageBlock` and triggers compression, then verify compression input no longer contains raw `DataBlock` entries and no crash occurs on formatter path.

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
backend
Issue 類型
缺陷
難度
2/5
預估耗時
1-3 小時
活躍度
冷清
描述清晰度
描述清楚
新手友好度
52/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。