cloudflare / cloudflare/agents
useAgentChat: recovered/resumed reply interleaves into scrambled text and never self-heals (clean server snapshot is discarded)
- Dominant language
- TypeScript
- Stars
- 5.6k
- Forks
- 711
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 53
Description
### Summary
Streamed assistant replies intermittently render as **scrambled characters** (e.g. `You can see the orders...` → `You can see Youthe or caders in sn the ee Admin...`), and stay scrambled until the page is reloaded. It happens across models and is a **client-side rendering** issue. The persisted transcript is always correct.
There are two parts:
1. **Trigger:** when a turn **recovers / resumes** mid-stream, the reply's chunks are delivered such that two copies interleave into a single message's text part. `applyChunkToParts` appends every `text-delta` to the *last* text part with no per-block/per-stream identity, so two overlapping copies merge chunk-by-chunk into gibberish.
2. **Why it never self-heals:** in `useAgentChat`, the `cf_agent_chat_messages` handler, while `streamStateRef` is `observing`, overrides the server's clean persisted snapshot with the polluted observing accumulator whenever `accumulator.parts.length >= snapshotParts`. So the clean copy the server broadcasts after `done` is discarded every time — only a reload (no observing accumulator) shows correct text.
### Environment
- Reproduced on `agents@0.21.0` (current latest). Also present in `0.20.1`.
- The non-heal override is in `packages/agents/src/chat/react.tsx` (`useAgentChat`, `cf_agent_chat_messages` case), the built bundle line is `if (observed.accumulator.parts.length >= snapshotParts) next = observed.accumulator.mergeInto(next);`.
- The resume reader `_createResumeStream` enqueues every frame for the request id with no per-replay dedup and closes on the first `done`.
- Client hook reached via `@cloudflare/ai-chat` → `agents/chat/react`; server harness `@cloudflare/think`.
### Minimal deterministic reproduction
Uses the real exported `StreamAccumulator`, no server needed.
```bash
mkdir agents-repro && cd agents-repro && npm init -y && npm i agents@0.21.0 ai@6 zod
# save the script below as repro.mjs, then:
node repro.mjs
```
```js
// repro.mjs
import { StreamAccumulator } from "agents/chat";
const REPLY = "You can see the orders in the Admin app.";
const toks = (s) => { const o = []; for (let i = 0; i < s.length; i += 3) o.push(s.slice(i, i + 3)); return o; };
const burst = (id) => [{ type: "start", messageId: id }, { type: "text-start" }, ...toks(REPLY).map((t) => ({ type: "text-delta", delta: t })), { type: "text-end" }, { type: "finish" }];
const text = (m) => m.parts.filter((p) => p.type === "text").map((p) => p.text).join("");
// Clean, server-persisted snapshot for message X (broadcast after `done`).
const clean = new StreamAccumulator({ messageId: "X" });
for (const c of burst("X")) clean.applyChunk(c);
const snapshot = [clean.toMessage()];
// Observing accumulator when a recovery replays the reply concurrently with the
// live stream: two copies interleave into one text part (uneven arrival rates).
const scrambled = new StreamAccumulator({ messageId: "X" });
{ const a = burst("X"), b = burst("X"); let i = 0, j = 0, s = 0;
while (i < a.length || j < b.length) { const A = s % 3 !== 2;
if (A && i < a.length) scrambled.applyChunk(a[i++]);
else if (j < b.length) scrambled.applyChunk(b[j++]);
else if (i < a.length) scrambled.applyChunk(a[i++]); s++; } }
// The cf_agent_chat_messages handler rule: observing accumulator overrides the
// snapshot whenever it has >= as many parts -> the clean server copy is discarded.
function currentSnapshotHandler(snapshot, acc) {
let next = [...snapshot];
const idx = next.findIndex((m) => m.id === acc.messageId);
const snapParts = idx >= 0 ? next[idx].parts.length : 0;
if (acc.parts.length >= snapParts) next = acc.mergeInto(next); // discards clean snapshot
return next;
}
console.log("server snapshot (clean):", JSON.stringify(text(snapshot[0])));
console.log("observing accumulator :", JSON.stringify(text(scrambled.toMessage())));
console.log("rendered after snapshot:", JSON.stringify(text(currentSnapshotHandler(snapshot, scrambled)[0])), "<-- scrambled; only a reload fixes it");
```
Output (fresh, unpatched `agents@0.21.0`):
```
server snapshot (clean): "You can see the orders in the Admin app."
observing accumulator : "You can see Youthe or caders in sn the ee Admin theapp. orders in the Admin app."
rendered after snapshot: "You can see Youthe or caders in sn the ee Admin theapp. orders in the Admin app." <-- scrambled; only a reload fixes it
```
### Expected
After the turn completes, the client should render the server's authoritative (clean) message, not a scrambled live accumulator.
### Two suggested fixes
- **Root cause (server/protocol):** on recovery/resume, don't deliver the reply so that two copies interleave into one message, a single replay per resumed stream, or fence a superseded generation before the recovery re-streams. Once two copies interleave into one text part they can't be separated client-side (no per-copy id on the deltas).
- **Client mitigation (heals it regardless of trigger):** in the `cf_agent_chat_messages` handler, only let the observing accumulator override the snapshot when it is a **forward extension** of it, i.e. also require `observedText.startsWith(snapshotText)` (concatenated text of `type === "text"` parts). A healthy live stream is always "snapshot + more" (kept); a scrambled/interleaved copy diverges (dropped → clean snapshot wins). This preserves every live-streaming path (cross-tab observe, healthy resume, brand-new streaming message) and heals the moment the reply is saved.
### Notes
`applyChunkToParts` appending `text-delta` to `findLastPartByType(parts, "text")` regardless of the chunk's own `id` is what lets two interleaved streams merge into one text part; making it id-aware would also help.
Contributor guide
Research direction
Start with packages/agents/src/chat/react.tsx, especially the useAgentChat cf_agent_chat_messages handler, then inspect StreamAccumulator.applyChunkToParts and _createResumeStream. Run the deterministic repro.mjs against agents@0.21.0 and compare the observing accumulator with the clean server snapshot. Done means a recovered reply no longer renders interleaved text and the authoritative post-done message remains visible without reloading.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend-api-design, frontend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 64/100