aws-samples / aws-samples/agents4energy

Backfill the beginning of an in-flight assistant message for late-joining viewers

Open
#170 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
48
Forks
68
PR merge metrics
No merged PRs in 30d

Description

### Background

Split out of the design discussion in #168 (comment: https://github.com/waltmayf/agentcore-amplify-fullstack/issues/66#issuecomment-4906466738).

The `/chat-handler` page now (as of that issue's latest round) supports a client subscribing to `onAgentEvent(sessionId)` **mid-stream** — it never saw that message's `text_message_start`, so it opens a bubble from the first `text_message_content` delta it receives and renders only the *tail* of the message (prefixed with `…` and flagged `missingStart: true` in `ChatMessage`). Once the message's `text_message_end`/`run_finished` fires, the client polls `listSessionMessages` (AgentCore memory) on a backoff until the authoritative full text is indexed, then swaps it in.

### The gap

Between "bubble opens with only the tail" and "poll succeeds and swaps in the full text" (up to ~15s per the current `POLL_DELAYS_MS` backoff in `web/app/(with-auth)/chat-handler/page.tsx`), a late-joining viewer sees an incomplete message with no way to know what was said before they connected — not great for someone opening a dashboard mid-run to "check on how the agent is doing."

### Proposal

Populate the *first part* of the incomplete message using context from the chat itself, rather than leaving it blank/missing until the poll resolves. Some options worth evaluating:

1. **Reconstruct from the prompt + prior turns.** The triggering user prompt (published as the `user_message` event) and the session's recent history/summary are already available to the joining client. An LLM call (or even a cheap heuristic) could produce a plausible-sounding "so far, the agent has likely said X" placeholder — clearly marked as reconstructed, not authoritative — until the real backfill lands.
2. **Persist a running snapshot server-side**, as discussed earlier in #168 (the "option 1" / `ActiveRun` model idea) — a throttled write of `accumulatedText` during the streaming loop in `agent.py::_run_agent`, read once on load before subscribing. This avoids guessing and gives the *real* prefix, at the cost of a new write path.
3. **Something in between**: keep the current poll-based backfill, but show a lightweight loading affordance (rather than a bare `…`-prefixed partial string) that makes it clear to the user that earlier content is still loading, not that the message actually started mid-sentence.

No implementation decision has been made yet — this issue is to track the follow-up and let a future pass evaluate 1 vs. 2 vs. 3 (or a combination).

---
*Imported from `waltmayf/agentcore-amplify-fullstack`#68 — originally filed by @app/github-actions on 2026-07-07. Migrated to open-source repo; cross-references updated post-import.*

Contributor guide

Open the contributing guide

Research direction

Start by reading web/app/(with-auth)/chat-handler/page.tsx, especially the POLL_DELAYS_MS backfill flow, and agent.py::_run_agent for the streaming loop. Compare the three approaches described in the issue and define one implementation path, including how reconstructed or persisted content is marked and how completion is verified when the authoritative message arrives.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, typescript
Domain
ai, full-stack
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.