Add a provider-neutral representation for streaming function-call updates
- Dominant language
- C#
- Stars
- 3.2k
- Forks
- 894
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 23
Description
### Background and motivation
AI providers can stream a function call's name and arguments incrementally. Microsoft.Extensions.AI currently exposes the completed call as `FunctionCallContent`, while incremental argument fragments are generally available only through provider-specific `ChatResponseUpdate.RawRepresentation`.
This requires middleware and UI integrations to understand each provider's native SDK. The same problem also occurs for protocol-backed `IChatClient` implementations: for example, an AG-UI client receives `TOOL_CALL_START` and `TOOL_CALL_ARGS` events but has no provider-neutral MEAI content type through which to expose them.
A common abstraction would allow provider clients and protocol adapters to represent the same lifecycle:
```text
provider or protocol update
-> FunctionCallUpdateContent
-> eventual completed FunctionCallContent
```
This would support predictive UI, MCP Apps, progress displays, logging, and other consumers that need to react while arguments are being generated.
Related scenario: ag-ui-protocol/ag-ui#2245.
### API Proposal
```csharp
namespace Microsoft.Extensions.AI;
public sealed class FunctionCallUpdateContent : AIContent
{
public FunctionCallUpdateContent(string callId);
public string CallId { get; }
public string? Name { get; set; }
public string? ArgumentsDelta { get; set; }
}
```
`FunctionCallUpdateContent` represents an incremental update only. `ArgumentsDelta` is an arbitrary text fragment and is not required to contain valid JSON.
Providers and protocol adapters would continue emitting the existing `FunctionCallContent` when the call is complete.
### API Usage
```csharp
await foreach (ChatResponseUpdate update in client.GetStreamingResponseAsync(messages))
{
foreach (var callUpdate in update.Contents.OfType())
{
UpdateToolProgress(
callUpdate.CallId,
callUpdate.Name,
callUpdate.ArgumentsDelta);
}
foreach (var call in update.Contents.OfType())
{
// The call is complete and its arguments are parsed.
}
}
```
A provider integration could emit:
```text
FunctionCallUpdateContent(call_1, name: "write_document")
FunctionCallUpdateContent(call_1, argumentsDelta: "{\"document\":\"Once")
FunctionCallUpdateContent(call_1, argumentsDelta: " upon a time\"}")
FunctionCallContent(call_1, "write_document", completeArguments)
```
`FunctionInvokingChatClient` should continue invoking only completed `FunctionCallContent` instances.
### Alternative Designs
**Continue using `RawRepresentation`.** This requires provider-specific casts and prevents middleware from working across providers and protocol-backed clients.
**Define an AG-UI-specific content type.** The concept is not specific to AG-UI; native provider clients expose the same incremental lifecycle.
**Expose only an accumulated partial JSON string.** Individual fragments more accurately represent provider behavior and allow consumers to choose their own accumulation or parsing strategy.
### Risks
- Consumers may incorrectly treat individual argument fragments as valid JSON.
- Providers differ in when they supply call IDs and function names.
- Parallel calls require providers to maintain stable call correlation.
- Aggregation must not cause a function to execute before the completed `FunctionCallContent` is available.
The new content should be additive and should not change existing coalescing or function-invocation behavior.
Contributor guide
Research direction
Start by reading the existing FunctionCallContent, AIContent, ChatResponseUpdate contents, and FunctionInvokingChatClient behavior described in the issue. Define the additive incremental content representation and verify that completed FunctionCallContent remains the only invocable form. Done means providers and protocol adapters can expose fragments without changing existing coalescing or invocation behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100