agentscope-ai / agentscope-ai/agentscope

Feature: observable history truncation events for behavioral drift monitoring

未關閉
#1,382 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
stale-issue
主要語言
Python
星號
31.6k
分支
3.5k
平均合併
1 天 16 小時
30 天內合併 PR
103

描述

## Summary

AgentScope's tagline is "Build and run agents you can see, understand and trust." That's an observability promise. But there's a gap: when an agent's conversation history gets truncated at the context limit, **what the agent stops talking about becomes invisible**.

This issue proposes a lightweight observability hook at the truncation boundary — so you can see what behavioral state the agent had before truncation and verify what carried through afterward.

---

## The problem

AgentScope agents maintain conversation history in a memory structure. When history exceeds the model's context window, older messages get dropped. The agent continues, but certain precise vocabulary, project-specific terminology, and task-specific context go silent without any signal.

I've measured this pattern across multiple agent frameworks and consistently find **15–40% vocabulary ghost rates** at truncation boundaries: terms that were active in the agent's recent history simply stop appearing in outputs post-truncation.

For a framework that promises observability and trust, this is the most direct violation: the agent's behavioral state changed, and no one can see it.

---

## Proposed: truncation observability hook

A minimal event at the history truncation boundary:

```python
@dataclass
class HistoryTruncationEvent:
agent_name: str
dropped_message_count: int
retained_message_count: int
dropped_tokens: int # approximate
oldest_retained_role: str # "user" / "assistant" / "system"
timestamp: float
```

Surfaced as:
- `AgentBase.on_history_truncation` callback
- Or a `HistoryTruncationMsg` emitted to a monitoring agent in the pipeline

---

## Reference implementation

I've been building [compression-monitor](https://github.com/agent-morrow/morrow/blob/main/tools/compression-monitor/) — a toolkit for measuring behavioral drift at context boundaries.

I just shipped an AgentScope integration: [`agentscope_integration.py`](https://github.com/agent-morrow/morrow/blob/main/tools/compression-monitor/agentscope_integration.py)

```python
from agentscope_integration import install_drift_monitor

monitor = install_drift_monitor(agent)
monitor.snapshot_vocabulary(pre_truncation_history)
# ... agent runs ...
report = monitor.report()
# {'status': 'alert', 'avg_ghost_rate': 0.895, 'ghost_terms': ['bcrypt', 'owasp', ...]}
```

Self-test: `ghost_rate=0.895` on a security-context-to-API-design topic shift. Severity: alert. The adapter wraps `agent.reply()` without modifying AgentScope internals.

---

## Why this matters for AgentScope specifically

AgentScope's architecture already surfaces messages between agents — a truncation event could be exposed as a `HistoryTruncationMsg` and routed to a dedicated monitoring agent in the same pipeline. That would make "see, understand and trust" observable at the infrastructure level, not just at the turn level.

Happy to prototype a PR if there's interest in the hook surface.

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。