Python: AG-UI + HistoryProvider: compose history+input without duplicating turns (UI vs LLM context authorities)

@eavanvalkenburg is already working on this.

Since Sep 8, 2026.

## Summary

When hosting AG-UI with a durable `HistoryProvider` (UI transcript vs LLM working memory as **two authorities**), the framework should **dedupe** when assembling model context (`history + run input`). Today that composition is a blind concatenate; hosts must invent wrappers (e.g. “latest user turn only”) to avoid stacking the same turns twice.

## Two authorities (intended split)

| Store | Authority | Role |
|---|---|---|
| **AG-UI thread snapshot** | UI transcript | Hydrate / refresh / Allow-Deny cards; client often re-sends a full `messages` list |
| **HistoryProvider** | LLM context | Prior turns loaded via `before_run` / per-service-call middleware into `SessionContext` |

This is a common production shape (also reflected in the `ag_ui_assistant_ui_chat` sample, which adds a host-level `LatestUserTurnAgent` to truncate AG-UI input before `Agent.run`).

## What the framework does today

1. **AG-UI adapter** (`run_agent_stream`) passes the reconstructed AG-UI message list straight into `agent.run(messages, …)` — no “latest turn only” truncation inside `agent_framework_ag_ui`.
2. **Agent core** builds model messages as `SessionContext.get_messages(include_input=True)` → **context (history) then `extend(input_messages)`** with **no** `filter_new_messages`.
3. **`filter_new_messages`** exists and is used on **persist** (`save_messages`) to avoid superlinear growth when replaying transcripts — it is **not** applied when composing what the model sees.
4. AG-UI **session continuation** deliberately **excludes** `HistoryProvider` source ids from serialized continuation state, so the *default* examples (no durable history) stay correct: empty history + full AG-UI messages each request. That default does **not** cover hosts that attach a durable, `load_messages=True` HistoryProvider.

## Failure mode

With durable HistoryProvider + AG-UI full (or turn-sized) input:

```text
HistoryProvider: [user, assistant(tool_call), …] # already saved (e.g. after an interrupt turn)
run input: [user, assistant(tool_call), tool(result), …] # same turn again from AG-UI / approval resolve
model sees: history + input → duplicated user / tool-call turns
```

HITL / approval resume makes this especially visible: the interrupt turn is often already in history, while Allow continues with the same user + assistant(tool_call) plus the new tool result.

Write-path dedupe may keep the DB clean; **the prompt still stacks**.

## Expected behavior

When AG-UI (or any caller) coexists with a HistoryProvider that loads prior turns:

- Model context should be equivalent to **`history + filter_new_messages(history, input)`** (or an equivalent overlap/suffix algorithm), so turns already present in history are not appended again from `input_messages`.
- Ideally this lives in one place (e.g. `get_messages(include_input=True)` and/or `PerServiceCallHistoryPersistingMiddleware` reconstruction), so every host does not reimplement truncation wrappers.
- Persist behavior can stay as today (`filter_new_messages` on save).

Optional / related: document the two-authority model explicitly for AG-UI hosts (snapshot = UI, HistoryProvider = LLM), and that relying on “empty history because continuation excludes history keys” is insufficient once a durable provider is attached.

## Suggested direction (non-prescriptive)

- Apply the existing `filter_new_messages(existing=history, incoming=input)` (or shared overlap logic) at **compose** time when `include_input=True` and history context is non-empty.
- Add a regression test: durable HistoryProvider already contains `[u, a]`; `run` input is `[u, a, tool]`; messages sent to the chat client are `[u, a, tool]` once — not `[u, a, u, a, tool]`.
- Keep default AG-UI examples (no durable history) unchanged.

## Environment

- Package: `agent-framework` / `agent-framework-ag-ui` (Python)
- Related host workaround pattern: sample `python/samples/05-end-to-end/ag_ui_assistant_ui_chat` (`LatestUserTurnAgent` truncates so HistoryProvider is not duplicated into the prompt)

## Related

- assistant-ui client resume re-sending full `messages` on Allow (worsens snapshot duplication on the UI path): https://github.com/assistant-ui/assistant-ui/issues/7011
That issue is about **snapshot** duplication; this issue is about **LLM context** duplication when HistoryProvider loads across turns.