ag-ui-protocol / ag-ui-protocol/ag-ui
[Feature] Client resends the full message history on every run — incremental sync for stateful backends
- 主要言語
- Python
- スター
- 15.9k
- フォーク
- 1.4k
- 平均マージ
- 1日 17時間
- マージ済み PR(30日)
- 163
説明
## Summary
On every run, `AbstractAgent.prepareRunAgentInput()` serializes the **entire** `this.messages` array into `RunAgentInput.messages` (it only strips `activity`-role messages — no bound on count). So a conversation with 1000 messages sends **1000 + 1** messages on the next turn, and 1001 + 1 on the one after that, and so on — an O(N) payload per turn to convey O(1) new content.
This is inherent to the base `AbstractAgent`, so it affects **every** transport (`HttpAgent` over SSE, WebSocket agents, etc.) and every integration (CopilotKit included).
## Why this matters for stateful backends
Many backends already persist the full conversation and rebuild the LLM context server-side. The ADK middleware (`ag-ui-adk`) is a concrete example: it holds sessions in a store and, on each run, `_get_unseen_messages()` **discards every message whose id is already in `processed_message_ids`** — i.e. it deliberately ignores the re-sent history and only processes the new message. The LLM context comes from the persisted session, not from the re-sent array.
So for these backends the client is uploading the whole history on every turn **only to have the server throw almost all of it away.** The costs are real even though the LLM/token cost is not doubled:
- **Bandwidth / latency**: the request body grows without bound; a 1000-message thread ships ~1000 messages of dead weight per turn.
- **Serialization / memory on the client**: `structuredClone_(this.messages)` on every run (see also #1644 — per-subscriber deep clones causing 20s+ main-thread blocks, and #1048 — O(N) cloning/serialization in long chats).
- **Mobile / metered networks**: uploading megabytes of already-known history per keystroke-turn is wasteful.
## Relationship to existing issues
- **#2062** (`messageFilter` on `AgentConfig`) is the closest — a client-side callback to prune messages before send. It's a good escape hatch, but it's a **manual, per-integration knob**: every app has to know to set it, and it has no notion of *what the server already has*. The integrator has to guess the filter.
- **#2159** asks for paginated message history on retrieval (`/agents/state`). This issue is the **upload** counterpart: the same unbounded-history problem, but on the outbound `RunAgentInput`.
## Proposed direction (for discussion)
A protocol-level notion of *incremental message sync*, so the client doesn't have to resend what the server already holds. A few shapes, from least to most invasive:
1. **Standardize "send only unconfirmed messages"** — the client tracks which message ids the server has acknowledged (e.g. via `MESSAGES_SNAPSHOT` / run completion) and, when the agent is known to be stateful, sends only messages not yet acknowledged. A capability flag (`GET /capabilities` → `statefulHistory: true`) could gate this so stateless backends keep receiving the full array.
2. **A sync cursor / `sinceMessageId`** on `RunAgentInput` — the client sends messages after a cursor the server previously returned; the server fills the rest from its store.
3. **Make `messageFilter` (#2062) first-class + defaulted** — ship it, and provide a built-in filter for the "stateful backend" case rather than leaving it to each integrator.
Each has trade-offs (spec change vs. client-only, correctness on reconnect/rewind, stateless-backend compatibility). I'd value a maintainer's take on whether the client resending full history is considered acceptable by design, or whether a standard incremental path is wanted.
## Repro (conceptual)
```ts
// After N turns, agent.messages.length === 2N (user+assistant).
// Every runAgent() call:
prepareRunAgentInput() // → RunAgentInput.messages = ALL 2N messages
// Backend (ag-ui-adk) _get_unseen_messages() → keeps only the 1 new one.
```
I'm happy to prototype option 1 or 3 (client-side, opt-in via capability) in `@ag-ui/client` + `ag-ui-adk` if that's a direction you'd accept.
コントリビューションガイド
評価
この issue はまだ評価されていません。