ag-ui-protocol / ag-ui-protocol/ag-ui

[Feature] Client resends the full message history on every run — incremental sync for stateful backends

Abierto
#2,186 2 comentarios 1 reacción 0 asignados Ver en GitHub
enhancement proposal
Lenguaje dominante
Python
Estrellas
15.9k
Forks
1.4k
Merge medio
1 d 17 h
PR fusionados (30 d)
163

Descripción

## Summary

On every run, `AbstractAgent.prepareRunAgentInput()` serializes the **entire** `this.messages` array into `RunAgentInput.messages` (it only strips `activity`-role messages — no bound on count). So a conversation with 1000 messages sends **1000 + 1** messages on the next turn, and 1001 + 1 on the one after that, and so on — an O(N) payload per turn to convey O(1) new content.

This is inherent to the base `AbstractAgent`, so it affects **every** transport (`HttpAgent` over SSE, WebSocket agents, etc.) and every integration (CopilotKit included).

## Why this matters for stateful backends

Many backends already persist the full conversation and rebuild the LLM context server-side. The ADK middleware (`ag-ui-adk`) is a concrete example: it holds sessions in a store and, on each run, `_get_unseen_messages()` **discards every message whose id is already in `processed_message_ids`** — i.e. it deliberately ignores the re-sent history and only processes the new message. The LLM context comes from the persisted session, not from the re-sent array.

So for these backends the client is uploading the whole history on every turn **only to have the server throw almost all of it away.** The costs are real even though the LLM/token cost is not doubled:

- **Bandwidth / latency**: the request body grows without bound; a 1000-message thread ships ~1000 messages of dead weight per turn.
- **Serialization / memory on the client**: `structuredClone_(this.messages)` on every run (see also #1644 — per-subscriber deep clones causing 20s+ main-thread blocks, and #1048 — O(N) cloning/serialization in long chats).
- **Mobile / metered networks**: uploading megabytes of already-known history per keystroke-turn is wasteful.

## Relationship to existing issues

- **#2062** (`messageFilter` on `AgentConfig`) is the closest — a client-side callback to prune messages before send. It's a good escape hatch, but it's a **manual, per-integration knob**: every app has to know to set it, and it has no notion of *what the server already has*. The integrator has to guess the filter.
- **#2159** asks for paginated message history on retrieval (`/agents/state`). This issue is the **upload** counterpart: the same unbounded-history problem, but on the outbound `RunAgentInput`.

## Proposed direction (for discussion)

A protocol-level notion of *incremental message sync*, so the client doesn't have to resend what the server already holds. A few shapes, from least to most invasive:

1. **Standardize "send only unconfirmed messages"** — the client tracks which message ids the server has acknowledged (e.g. via `MESSAGES_SNAPSHOT` / run completion) and, when the agent is known to be stateful, sends only messages not yet acknowledged. A capability flag (`GET /capabilities` → `statefulHistory: true`) could gate this so stateless backends keep receiving the full array.
2. **A sync cursor / `sinceMessageId`** on `RunAgentInput` — the client sends messages after a cursor the server previously returned; the server fills the rest from its store.
3. **Make `messageFilter` (#2062) first-class + defaulted** — ship it, and provide a built-in filter for the "stateful backend" case rather than leaving it to each integrator.

Each has trade-offs (spec change vs. client-only, correctness on reconnect/rewind, stateless-backend compatibility). I'd value a maintainer's take on whether the client resending full history is considered acceptable by design, or whether a standard incremental path is wanted.

## Repro (conceptual)

```ts
// After N turns, agent.messages.length === 2N (user+assistant).
// Every runAgent() call:
prepareRunAgentInput() // → RunAgentInput.messages = ALL 2N messages
// Backend (ag-ui-adk) _get_unseen_messages() → keeps only the 1 new one.
```

I'm happy to prototype option 1 or 3 (client-side, opt-in via capability) in `@ag-ui/client` + `ag-ui-adk` if that's a direction you'd accept.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.