Chat client re-sends the full message transcript on every settle (O(n²) reporting)

Summary

The chat client reports its entire rendered message transcript to the server every time a message settles. Over a long conversation this sends roughly O(n²) bytes across a session, and each snapshot can be large because it also carries retained HTML dependencies. We should reduce this, likely via an incremental (delta/append) report protocol — but the fix has an architectural trade-off worth discussing before implementing.

Background: how message reporting works today

Since the chat UI moved to React, the browser is the source of truth for the user-facing conversation (what's actually displayed, including any app-applied transformations). The server no longer keeps its own copy of displayed messages; instead the client reports them and the server treats that report as authoritative.

Mechanically:

• Whenever the set of settled (non-streaming) messages changes, the client serializes all of them and sends the result to the server as a Shiny input, input$<id>_messages (tagged shinychat.messages).
  • Builder: buildMessagesSnapshot() in js/src/chat/state.ts — maps every settled message to { role, segments, attachments?, htmlDeps? }.
  • Trigger: the reportSnapshot effect in js/src/chat/ChatApp.tsx (fires on state.messages changes) and a manual co-send on submit.
  • Transport: sendMessagesSnapshot() in js/src/transport/shiny-transport.ts.
• The server consumes the report as the full current list:
  • Python — Chat._reported_messages() / Chat.messages() in pkg-py/src/shinychat/_chat.py, plus bookmark serialization and history, read the input directly.
  • R — get_reported_messages() in pkg-r/R/chat_history.R; history's on_response() takes the full reported list and slices the newly-added tail using a ui_offset cursor before attaching UI to record nodes.

Streaming does not trigger a report per chunk (in-flight content lives in a separate state.streamingMessage field), so the report fires roughly once per settle point — not on every token. The cost below is about settle-point frequency, not streaming.

The problem

Every settle point re-sends the whole transcript:

• Turn 1 completes → send 1 message.
• Turn 2 completes → send 2 messages.
• …
• Turn n completes → send n messages.

Total ≈ 1 + 2 + … + n = O(n²) message-payloads over the session. Each payload is not tiny either: segments hold the full rendered content string of each message, and any message that embedded a widget/HTML output carries its htmlDeps (script/stylesheet references), which are re-sent inside every subsequent snapshot.

For typical short conversations this is fine. For long-lived or widget-heavy chats it means steadily growing per-message websocket traffic and server-side deserialization work that scales with the square of the conversation length.

Why it isn't a trivial fix

The obvious fix — have the client send only what's new since its last report — requires the server to accumulate reports into the current list. That reintroduces server-side state that the client-authoritative design deliberately removed (see #272, which deleted Python's _store_message accumulator). In particular, Chat.messages() currently returns the input value as-is; with a delta protocol it would have to read from an accumulator instead.

So this is a genuine design decision, not just an optimization: is a small, dumb append buffer on the server acceptable, given the goal of keeping the browser as the single source of truth?

Options to explore

1. Append-delta protocol. Client sends only messages appended since its last acknowledged report, plus a base count; server maintains a thin append buffer and exposes the accumulated list to .messages() / history. Simplest incremental model; reintroduces a minimal accumulator.
2. Content-addressed messages. Give each message a stable id; unchanged messages are sent as { id } references rather than full bodies; the server merges by id into its record. Avoids re-sending bodies without a strict append assumption (tolerates edits/branching), at the cost of a merge step.
3. Session-level htmlDeps table. Deduplicate HTML dependencies across the session — send each dep once and reference it by index from messages. Orthogonal to 1/2 and stackable; helps the widget-heavy case specifically.
4. Do nothing / cap. Accept the ceiling and instead document it, or cap transcript length. Lowest effort; punts the scaling problem.

Acceptance criteria

• Per-session client→server bytes for reporting grow ~linearly (not quadratically) with conversation length.
• Chat.messages() (Python), bookmark serialization, and R history restore continue to reflect exactly what was displayed, including app-applied transformations.
• Behavior is consistent across the Python and R implementations.

Context

Raised in review of #272 (making the browser the source of truth for chat messages). That PR intentionally left this as a follow-up to keep its scope focused on the source-of-truth change.

Start with buildMessagesSnapshot() in js/src/chat/state.ts, the reportSnapshot effect in js/src/chat/ChatApp.tsx, and sendMessagesSnapshot() in js/src/transport/shiny-transport.ts. Then trace Python’s Chat._reported_messages()/Chat.messages() in pkg-py/src/shinychat/_chat.py and R’s get_reported_messages() in pkg-r/R/chat_history.R. Done means reporting grows roughly linearly while Python bookmarks and R history still restore exactly what was displayed.