cloudflare / cloudflare/agents

sessions: the change-feed cache mirror is duplicated in every host

Open
#2,205 0 comments 0 reactions 1 assignee Claimed by @threepointone View on GitHub
enhancement
Dominant language
TypeScript
Stars
5.6k
Forks
711
Avg merge
1d 20h
Merged PRs (30d)
53

Description

Every host that keeps an in-memory transcript reimplements the same reduction over `sessions.subscribe()`. Think and AIChatAgent each carry their own copy today, and pi will need a third.

## The duplication

Both hosts write the same switch:

| event | what both do |
| --- | --- |
| `append` | find by id; replace if present, push if not |
| `update` | find by id; replace in place, ignore if absent |
| `delete` | filter out the removed ids |
| `clear` | empty the array |

plus their own `findIndex`-based upsert and patch helpers around it. `Think._upsertCachedMessage` / `_patchCachedMessage` / `_replaceCachedMessages` and the equivalent inline blocks in `AIChatAgent` are the same code with different field names.

This is a reduction over an ordered change feed, which is exactly the kind of thing the capability that emits the feed should be able to do for you.

## Why it has not moved yet

The differences between the two are real, not incidental, and a naive shared implementation would be wrong for both:

- **Think resyncs where AIChat patches.** A branch append (`event.parentId !== undefined`) and a `compact` event both make Think do a full `_syncMessages()`, because the path itself changed and an in-place patch cannot express that. It also refreshes its system prompt on compact. AIChat has no branch model and no prompt to refresh.
- **AIChat's cache is public API.** `this.messages` is a documented field that existing subclasses assign to directly. It cannot become an opaque handle without a breaking change.
- **Coherence during streaming is load-bearing.** The comments around `_applyToolUpdateToMessages` record a real bug — a full re-read mid-turn drops in-flight messages whose parent chain is not yet persisted — and the fix was specifically to patch in place rather than resync. Any shared mirror has to preserve that, and getting it wrong reintroduces a bug that took two reverts to settle.

So the shape is not simply "move the switch down". It needs a seam that lets a host choose patch-vs-resync per event, and lets AIChat keep a plain array it owns.

## Sketch

Something like a `session.mirror(target, hooks)` that owns the reduction and calls back for the decisions only the host can make:

```ts
const mirror = session.mirror({
onResyncNeeded: () => this._syncMessages(), // Think: branch append, compact
transform: (m) => autoTransformMessages([m])[0] // AIChat: v4 -> v5
})
```

The transform hook matters: AIChat's `#messageForCache` does version transformation, which is genuinely its own concern and must stay a host decision rather than something sessions knows about.

## Prior art in this PR

#2196 already removed one instance of hosts working around the feed. `appendMessage` used to return `getMessageRaw` on its duplicate paths, so the feed emitted pointer-form messages on some appends and inline messages on others. Think compensated by serializing every incoming message and substring-searching it for `attachment:sha256:` before deciding whether to re-read — on the streaming hot path. Making the feed emit one consistent shape deleted that helper outright.

That is the same category of problem: when the feed's contract is not quite enough, every host grows its own patch. Worth doing the mirror properly rather than waiting for a third copy to appear in pi.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.