cloudflare / cloudflare/agents

useAgentChat: recovered/resumed reply interleaves into scrambled text and never self-heals (clean server snapshot is discarded)

Open
#2,166 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
5.6k
Forks
711
Avg merge
1d 20h
Merged PRs (30d)
53

Description

### Summary

Streamed assistant replies intermittently render as **scrambled characters** (e.g. `You can see the orders...` → `You can see Youthe or caders in sn the ee Admin...`), and stay scrambled until the page is reloaded. It happens across models and is a **client-side rendering** issue. The persisted transcript is always correct.

There are two parts:

1. **Trigger:** when a turn **recovers / resumes** mid-stream, the reply's chunks are delivered such that two copies interleave into a single message's text part. `applyChunkToParts` appends every `text-delta` to the *last* text part with no per-block/per-stream identity, so two overlapping copies merge chunk-by-chunk into gibberish.
2. **Why it never self-heals:** in `useAgentChat`, the `cf_agent_chat_messages` handler, while `streamStateRef` is `observing`, overrides the server's clean persisted snapshot with the polluted observing accumulator whenever `accumulator.parts.length >= snapshotParts`. So the clean copy the server broadcasts after `done` is discarded every time — only a reload (no observing accumulator) shows correct text.

### Environment

- Reproduced on `agents@0.21.0` (current latest). Also present in `0.20.1`.
- The non-heal override is in `packages/agents/src/chat/react.tsx` (`useAgentChat`, `cf_agent_chat_messages` case), the built bundle line is `if (observed.accumulator.parts.length >= snapshotParts) next = observed.accumulator.mergeInto(next);`.
- The resume reader `_createResumeStream` enqueues every frame for the request id with no per-replay dedup and closes on the first `done`.
- Client hook reached via `@cloudflare/ai-chat` → `agents/chat/react`; server harness `@cloudflare/think`.

### Minimal deterministic reproduction

Uses the real exported `StreamAccumulator`, no server needed.

```bash
mkdir agents-repro && cd agents-repro && npm init -y && npm i agents@0.21.0 ai@6 zod
# save the script below as repro.mjs, then:
node repro.mjs
```

```js
// repro.mjs
import { StreamAccumulator } from "agents/chat";

const REPLY = "You can see the orders in the Admin app.";
const toks = (s) => { const o = []; for (let i = 0; i < s.length; i += 3) o.push(s.slice(i, i + 3)); return o; };
const burst = (id) => [{ type: "start", messageId: id }, { type: "text-start" }, ...toks(REPLY).map((t) => ({ type: "text-delta", delta: t })), { type: "text-end" }, { type: "finish" }];
const text = (m) => m.parts.filter((p) => p.type === "text").map((p) => p.text).join("");

// Clean, server-persisted snapshot for message X (broadcast after `done`).
const clean = new StreamAccumulator({ messageId: "X" });
for (const c of burst("X")) clean.applyChunk(c);
const snapshot = [clean.toMessage()];

// Observing accumulator when a recovery replays the reply concurrently with the
// live stream: two copies interleave into one text part (uneven arrival rates).
const scrambled = new StreamAccumulator({ messageId: "X" });
{ const a = burst("X"), b = burst("X"); let i = 0, j = 0, s = 0;
while (i < a.length || j < b.length) { const A = s % 3 !== 2;
if (A && i < a.length) scrambled.applyChunk(a[i++]);
else if (j < b.length) scrambled.applyChunk(b[j++]);
else if (i < a.length) scrambled.applyChunk(a[i++]); s++; } }

// The cf_agent_chat_messages handler rule: observing accumulator overrides the
// snapshot whenever it has >= as many parts -> the clean server copy is discarded.
function currentSnapshotHandler(snapshot, acc) {
let next = [...snapshot];
const idx = next.findIndex((m) => m.id === acc.messageId);
const snapParts = idx >= 0 ? next[idx].parts.length : 0;
if (acc.parts.length >= snapParts) next = acc.mergeInto(next); // discards clean snapshot
return next;
}

console.log("server snapshot (clean):", JSON.stringify(text(snapshot[0])));
console.log("observing accumulator :", JSON.stringify(text(scrambled.toMessage())));
console.log("rendered after snapshot:", JSON.stringify(text(currentSnapshotHandler(snapshot, scrambled)[0])), "<-- scrambled; only a reload fixes it");
```

Output (fresh, unpatched `agents@0.21.0`):

```
server snapshot (clean): "You can see the orders in the Admin app."
observing accumulator : "You can see Youthe or caders in sn the ee Admin theapp. orders in the Admin app."
rendered after snapshot: "You can see Youthe or caders in sn the ee Admin theapp. orders in the Admin app." <-- scrambled; only a reload fixes it
```

### Expected

After the turn completes, the client should render the server's authoritative (clean) message, not a scrambled live accumulator.

### Two suggested fixes

- **Root cause (server/protocol):** on recovery/resume, don't deliver the reply so that two copies interleave into one message, a single replay per resumed stream, or fence a superseded generation before the recovery re-streams. Once two copies interleave into one text part they can't be separated client-side (no per-copy id on the deltas).
- **Client mitigation (heals it regardless of trigger):** in the `cf_agent_chat_messages` handler, only let the observing accumulator override the snapshot when it is a **forward extension** of it, i.e. also require `observedText.startsWith(snapshotText)` (concatenated text of `type === "text"` parts). A healthy live stream is always "snapshot + more" (kept); a scrambled/interleaved copy diverges (dropped → clean snapshot wins). This preserves every live-streaming path (cross-tab observe, healthy resume, brand-new streaming message) and heals the moment the reply is saved.

### Notes

`applyChunkToParts` appending `text-delta` to `findLastPartByType(parts, "text")` regardless of the chunk's own `id` is what lets two interleaved streams merge into one text part; making it id-aware would also help.

Contributor guide

Open the contributing guide

Research direction

Start with packages/agents/src/chat/react.tsx, especially the useAgentChat cf_agent_chat_messages handler, then inspect StreamAccumulator.applyChunkToParts and _createResumeStream. Run the deterministic repro.mjs against agents@0.21.0 and compare the observing accumulator with the clean server snapshot. Done means a recovered reply no longer renders interleaved text and the authoritative post-done message remains visible without reloading.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend-api-design, frontend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.