ag-ui-protocol / ag-ui-protocol/ag-ui

[Bug]: @ag-ui/langgraph: unconditional RAW + per-event STATE_SNAPSHOT make the stream O(state × events) — 42× amplification, no opt-out

Đang mở
#2,600 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
15.9k
Fork
1.4k
Merge trung bình
1 ngày 17 giờ
Pull request đã merge (30 ngày)
163

Mô tả

### Pre-flight Checklist
- [x] I have searched existing issues and this hasn't been reported yet.
- [x] I am using the latest version of AG-UI (`@ag-ui/langgraph@0.0.43`).

### Describe the Bug

`LangGraphAgent` re-emits the graph's full state and the raw LangGraph event
firehose on nearly every event, with no way to turn either off. On a
one-tool agent whose tool returns a 33.9 KB payload **once**, the AG-UI SSE
stream to the client is **1,444 KB and contains 38 copies of that payload**.

Four compounding behaviors (all in `LangGraphAgent`, verified against 0.0.43):

1. **`RAW` is dispatched unconditionally** for every LangGraph event
(`this.dispatchEvent({type: RAW, event})` in `handleStreamEvents`).
`on_chain_start`/`on_chat_model_start` inputs carry the full message
history — including the system prompt — so prompts, tool payloads, and
message history are broadcast to the browser on every model call. No flag
gates this. (The ADK adapter never emits RAW; #2098 proposes it as
*opt-in* there. The .NET SDK's equivalent is #2297.)

2. **`rawEvent` is stapled onto most translated events** (STATE_SNAPSHOT,
TEXT_MESSAGE_*, TOOL_CALL_*), duplicating each raw event a second time on
the wire. Same shape as #2297 (.NET), where it measured 66% of the wire.

3. **`STATE_SNAPSHOT` carries the entire graph state**, and emission is
decided by `JSON.stringify(currentState) !== JSON.stringify(prevState)`
executed **per incoming event** — two full-state serializations per event
(CPU), and the full state on the wire every time anything changed. With a
generated artifact (an HTML page, a base64 image) in state or messages,
every snapshot re-ships it.

4. **`updates` mode is requested and then discarded**: the default
`streamMode` is `["events","values","updates","messages-tuple"]`, but the
handler does `if (event === "updates") continue`. The backend pays to
produce and ship a mode the adapter drops on the floor.

There is **no escape hatch**: overriding `forwardedProps.streamMode` to drop
`events` produces an empty AG-UI stream, because translation only happens
from `events`-mode callbacks — and the `messages-tuple` path is broken by two
separate bugs (filed separately: #2601). So every user is forced
onto the maximal-duplication configuration.

Client-side filtering (the resolution of #141) does not help: the server has
already materialized and serialized everything (our production deployment
OOMs server-side), the bytes are already on the wire, and the
prompt/payload exposure has already happened.

### Steps to Reproduce

Minimal project (from the official docs): a LangGraph agent with **one tool**
that returns a ~34 KB HTML page containing a unique marker, served by
`langgraphjs dev`, fronted by CopilotKit runtime v2 + `LangGraphAgent`
(express example from the package README). Drive one run and count marker
copies per AG-UI event type.

```ts
// graph.ts — the entire agent
const loadPage = tool(async () => PAGE /* 33.9 KB, unique marker */, {
name: "load_page", description: "Return the full landing page HTML.",
schema: z.object({}),
});
export const agent = createAgent({ llm, tools: [loadPage],
prompt: "Call load_page exactly once. Do NOT repeat its output. Then reply: done" });
```

```js
// runtime.mjs — straight from the docs
const runtime = new CopilotRuntime({
agents: { agent: new LangGraphAgent({ deploymentUrl: "http://localhost:2024", graphId: "agent" }) },
});
app.use(createCopilotExpressHandler({ runtime, basePath: "/api/copilotkit" }));
```

POST one message to `/api/copilotkit/agent/agent/run` and measure the SSE.

### Results

| forwardedProps.streamMode | SSE bytes | copies of the 33.9 KB payload | assistant text |
|---|---|---|---|
| default (none sent) | **1,444 KB** | **38** | "done" ✅ |
| `["events"]` | 1,210 KB | 32 | "done" ✅ |
| `["values","updates","messages-tuple"]` | 189 KB | 5 | **empty** ❌ |
| `["messages-tuple"]` | 153 KB | 4 | **empty** ❌ |

Default-mode split: `STATE_SNAPSHOT` 930.8 KB (25 payload copies), `RAW`
401.0 KB (10 copies), `TOOL_CALL_RESULT` 69.2 KB (2), `MESSAGES_SNAPSHOT`
34.5 KB (1). The real content of the entire turn is ~34 KB.

That the amplification comes from `events` mode itself is reproducible
without AG-UI: on a 4-node graph with no LLM, `streamEvents` yields 445 KB
for 49 KB of state (9.1×) while `stream({modes:["updates","messages"]})`
yields 49 KB (1.0×).

### Expected Behavior

Config to opt out of the duplication, e.g.:

- `emitRawEvents?: boolean` (default true for compat) — gate the RAW dispatch;
- `includeRawEvent?: boolean` — stop stapling `rawEvent` onto translated events;
- a snapshot policy — emit `STATE_SNAPSHOT` on node boundaries / run end only,
or an allowlist/denylist of state keys;
- don't request `streamMode: updates` while unconditionally discarding it.

### Environment

- `@ag-ui/langgraph` 0.0.43, `@copilotkit/runtime` 1.69.3 (v2 endpoints)
- `@langchain/langgraph` 1.4.13, sdk 1.10.0, core 1.2.9, `langgraphjs-cli` 1.4.5 (`langgraphjs dev`)
- Node 20 (graph server) / Node 25 (runtime), macOS

### Additional Context (production impact)

- CopilotKit runtime hosted on Cloudflare Workers (hard 128 MB isolate heap):
a landing-page-builder agent whose real output is a ~2 MB HTML page
produces a **457 MB** run stream; a heap snapshot mid-run shows **158
retained copies of the page HTML** and the worker dies with
`exceededMemory`. There is no configuration that avoids this short of
forking the adapter.
- A second, unrelated production agent (plain LangGraph router, OpenAI
models) shows the same signature on a *greeting*: 1.2 MB / 443 events for a
117-char reply — the user's one-word message appears **214×**, the 52 KB
system prompt is shipped **4×**, `STATE_SNAPSHOT` 56.7% + `RAW` 37.6% of
the wire.
- Related: #2297 (.NET rawEvent opt-out — same measurement, different SDK),
#2098 (ADK: RAW as opt-in), #141 (client-side filtering doesn't address
server memory / wire / exposure), #883 (large streams also hurt the client).

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.