ag-ui-protocol / ag-ui-protocol/ag-ui

[.NET] Predictive state updates: stream a tool call's arguments to the UI as the model writes them

Aberta
#2,245 1 comentário 1 reação 1 responsável Reivindicada por @ranst91 Ver no GitHub
Linguagem predominante
Python
Estrelas
15.9k
Forks
1.4k
Merge médio
1d 17h
PRs com merge (30d)
163

Descrição

> **Note (edited 2026-08-02):** Rewritten to lead with the scenario rather than a specific API shape, and to correct inaccuracies in the original filing.

**Area:** .NET SDK — `sdks/dotnet` (primarily `AGUI.Server`)

## Scenario

A chat panel sits beside a document editor. The user asks for a short story; the model calls `write_document`.

- **Expected:** the story appears in the editor a few words at a time as the model writes it, then an approve/reject prompt appears.
- **Actual (.NET):** the editor stays empty for the whole generation, then the document appears at once.

AG-UI calls this **predictive state updates** — optimistically rendering a tool call's *arguments* while they're being generated, before the tool runs. It generalizes to any surface outside the chat transcript: forms filling in field by field, code appearing in an editor, a spreadsheet or itinerary being drafted, or simply rendering a pending tool call's arguments instead of a spinner. LangGraph, CrewAI, ADK, Strands and the reference Python server all support it, and it's one of the dojo's headline scenarios.

## Current behavior

`Microsoft.Extensions.AI` coalesces a provider's argument deltas into a single completed `FunctionCallContent` before `AGUI.Server` sees it, so one `TOOL_CALL_ARGS` event carries the whole value. Measured with a local probe (shipped packages, no credentials — available on request): against a provider streaming a document over ~300 ms, the argument surfaced **once, complete, after generation finished**.

`AGUIStreamOptions.MapCall(toolName, ...)` is therefore also called once with the finished arguments. Anything built on it re-chunks a value it already holds in full — which is what the dojo's `predictive_state_updates` sample does, and why its output arrives all at once.

The raw-event escape hatch doesn't close this. `AGUI.Server` will emit any `BaseEvent` supplied via `RawRepresentation`, so emitting paced `TOOL_CALL_ARGS` is not the obstacle — the deltas are already gone by the time app code runs. The only workaround is provider-specific: reading `FunctionArgumentsUpdate` off `RawRepresentation` in a `DelegatingChatClient`.

**The client side already works.** A custom `IAGUITransport` observes the raw `TOOL_CALL_ARGS` stream, and predictive state is achievable that way today. That half is a papercut, not a gap (see below).

## Proposal

A declarative call-side mapping, so app code does no chunking, no partial-JSON handling, and no hand-rolled state events:

```csharp
var streamOptions = new AGUIStreamOptions();

// Existing, result-side: "when write_document returns, that's the new state."
streamOptions.MapResultAsStateSnapshot("write_document");

// Proposed, call-side: "while the model is writing write_document's `document` argument,
// that argument *is* the `document` state — show it as it arrives."
streamOptions.PredictState(stateKey: "document", toolName: "write_document", argument: "document");

app.MapAGUIServer("/predictive_state_updates", agent)
.WithMetadata(streamOptions);
```

Two desired properties:

1. **Declarative** — no manual chunking or partial-JSON handling in app code.
2. **The tool still runs normally** — prediction is observational, so it shouldn't require intercepting or suppressing the tool call, or disabling function invocation, and shouldn't care whether the tool is frontend or backend.

**Open question on prior art.** LangGraph, CrewAI, ADK and Strands declare this mapping by emitting a `CUSTOM` event named `PredictState` (`{state_key, tool, tool_argument}`), consumed by the JS client's legacy CopilotKit compatibility layer — the path the dojo runs on today. The non-legacy `AgentSubscriber.onToolCallArgsEvent` (which already surfaces `partialToolCallArgs`) looks like the more natural fit for a new SDK, and a server-side declaration needn't travel over the wire at all.

## Client-side papercuts

Both optional and separable from the main ask:

- **`AGUIHttpTransport` is `internal`**, so it can't be wrapped. Observing the raw event stream requires re-implementing the HTTP transport (POST + `ReadAGUIEventStreamAsync`) purely to add a passthrough — ~20 lines of copied plumbing.
- **No partial-JSON helper.** Accumulating deltas is easy; turning a truncated `{"document":"Once upon a ti` into a usable value is not. The JS client uses `untruncate-json` and exposes the result as `partialToolCallArgs`; a .NET equivalent would save every consumer hand-rolling quote/brace balancing.

## Upstream dependency

The server-side fix needs argument deltas to reach `AGUI.Server`. MEAI's argument coalescing behavior prevents this. Coalescing is the right default for *invoking* a function, so this likely wants an opt-in rather than a behavior change. Consider tracking an optional non-coalescing behavior in dotnet/extensions.

Related: microsoft/agent-framework#4177 covers the `StateBag` half of state emission and also touches streaming tool-argument deltas.

@javiercn

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.