ag-ui-protocol / ag-ui-protocol/ag-ui

[Feature] Client resends the full message history on every run — incremental sync for stateful backends

Đang mở
#2,186 2 bình luận 1 reaction 0 người được giao Xem trên GitHub
enhancement proposal
Ngôn ngữ chính
Python
Star
15.9k
Fork
1.4k
Merge trung bình
1 ngày 17 giờ
Pull request đã merge (30 ngày)
163

Mô tả

## Summary

On every run, `AbstractAgent.prepareRunAgentInput()` serializes the **entire** `this.messages` array into `RunAgentInput.messages` (it only strips `activity`-role messages — no bound on count). So a conversation with 1000 messages sends **1000 + 1** messages on the next turn, and 1001 + 1 on the one after that, and so on — an O(N) payload per turn to convey O(1) new content.

This is inherent to the base `AbstractAgent`, so it affects **every** transport (`HttpAgent` over SSE, WebSocket agents, etc.) and every integration (CopilotKit included).

## Why this matters for stateful backends

Many backends already persist the full conversation and rebuild the LLM context server-side. The ADK middleware (`ag-ui-adk`) is a concrete example: it holds sessions in a store and, on each run, `_get_unseen_messages()` **discards every message whose id is already in `processed_message_ids`** — i.e. it deliberately ignores the re-sent history and only processes the new message. The LLM context comes from the persisted session, not from the re-sent array.

So for these backends the client is uploading the whole history on every turn **only to have the server throw almost all of it away.** The costs are real even though the LLM/token cost is not doubled:

- **Bandwidth / latency**: the request body grows without bound; a 1000-message thread ships ~1000 messages of dead weight per turn.
- **Serialization / memory on the client**: `structuredClone_(this.messages)` on every run (see also #1644 — per-subscriber deep clones causing 20s+ main-thread blocks, and #1048 — O(N) cloning/serialization in long chats).
- **Mobile / metered networks**: uploading megabytes of already-known history per keystroke-turn is wasteful.

## Relationship to existing issues

- **#2062** (`messageFilter` on `AgentConfig`) is the closest — a client-side callback to prune messages before send. It's a good escape hatch, but it's a **manual, per-integration knob**: every app has to know to set it, and it has no notion of *what the server already has*. The integrator has to guess the filter.
- **#2159** asks for paginated message history on retrieval (`/agents/state`). This issue is the **upload** counterpart: the same unbounded-history problem, but on the outbound `RunAgentInput`.

## Proposed direction (for discussion)

A protocol-level notion of *incremental message sync*, so the client doesn't have to resend what the server already holds. A few shapes, from least to most invasive:

1. **Standardize "send only unconfirmed messages"** — the client tracks which message ids the server has acknowledged (e.g. via `MESSAGES_SNAPSHOT` / run completion) and, when the agent is known to be stateful, sends only messages not yet acknowledged. A capability flag (`GET /capabilities` → `statefulHistory: true`) could gate this so stateless backends keep receiving the full array.
2. **A sync cursor / `sinceMessageId`** on `RunAgentInput` — the client sends messages after a cursor the server previously returned; the server fills the rest from its store.
3. **Make `messageFilter` (#2062) first-class + defaulted** — ship it, and provide a built-in filter for the "stateful backend" case rather than leaving it to each integrator.

Each has trade-offs (spec change vs. client-only, correctness on reconnect/rewind, stateless-backend compatibility). I'd value a maintainer's take on whether the client resending full history is considered acceptable by design, or whether a standard incremental path is wanted.

## Repro (conceptual)

```ts
// After N turns, agent.messages.length === 2N (user+assistant).
// Every runAgent() call:
prepareRunAgentInput() // → RunAgentInput.messages = ALL 2N messages
// Backend (ag-ui-adk) _get_unseen_messages() → keeps only the 1 new one.
```

I'm happy to prototype option 1 or 3 (client-side, opt-in via capability) in `@ag-ui/client` + `ag-ui-adk` if that's a direction you'd accept.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.