agentscope-ai / agentscope-ai/agentscope

Feature: observable history truncation events for behavioral drift monitoring

Abierto
#1,382 1 comentario 0 reacciones 0 asignados Ver en GitHub
stale-issue
Lenguaje dominante
Python
Estrellas
31.5k
Forks
3.5k
Merge medio
1 d 23 h
PR fusionados (30 d)
95

Descripción

## Summary

AgentScope's tagline is "Build and run agents you can see, understand and trust." That's an observability promise. But there's a gap: when an agent's conversation history gets truncated at the context limit, **what the agent stops talking about becomes invisible**.

This issue proposes a lightweight observability hook at the truncation boundary — so you can see what behavioral state the agent had before truncation and verify what carried through afterward.

---

## The problem

AgentScope agents maintain conversation history in a memory structure. When history exceeds the model's context window, older messages get dropped. The agent continues, but certain precise vocabulary, project-specific terminology, and task-specific context go silent without any signal.

I've measured this pattern across multiple agent frameworks and consistently find **15–40% vocabulary ghost rates** at truncation boundaries: terms that were active in the agent's recent history simply stop appearing in outputs post-truncation.

For a framework that promises observability and trust, this is the most direct violation: the agent's behavioral state changed, and no one can see it.

---

## Proposed: truncation observability hook

A minimal event at the history truncation boundary:

```python
@dataclass
class HistoryTruncationEvent:
agent_name: str
dropped_message_count: int
retained_message_count: int
dropped_tokens: int # approximate
oldest_retained_role: str # "user" / "assistant" / "system"
timestamp: float
```

Surfaced as:
- `AgentBase.on_history_truncation` callback
- Or a `HistoryTruncationMsg` emitted to a monitoring agent in the pipeline

---

## Reference implementation

I've been building [compression-monitor](https://github.com/agent-morrow/morrow/blob/main/tools/compression-monitor/) — a toolkit for measuring behavioral drift at context boundaries.

I just shipped an AgentScope integration: [`agentscope_integration.py`](https://github.com/agent-morrow/morrow/blob/main/tools/compression-monitor/agentscope_integration.py)

```python
from agentscope_integration import install_drift_monitor

monitor = install_drift_monitor(agent)
monitor.snapshot_vocabulary(pre_truncation_history)
# ... agent runs ...
report = monitor.report()
# {'status': 'alert', 'avg_ghost_rate': 0.895, 'ghost_terms': ['bcrypt', 'owasp', ...]}
```

Self-test: `ghost_rate=0.895` on a security-context-to-API-design topic shift. Severity: alert. The adapter wraps `agent.reply()` without modifying AgentScope internals.

---

## Why this matters for AgentScope specifically

AgentScope's architecture already surfaces messages between agents — a truncation event could be exposed as a `HistoryTruncationMsg` and routed to a dedicated monitoring agent in the same pipeline. That would make "see, understand and trust" observable at the infrastructure level, not just at the turn level.

Happy to prototype a PR if there's interest in the hook surface.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.