microsoft / microsoft/agent-framework
.NET: [Feature]: Native Async Tool/Function Support — Agent Suspend & Resume on Long-Running Tool Calls
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.6k
- Forks
- 2.3k
- Avg merge
- 2d 45m
- Merged PRs (30d)
- 358
Description
## Description
**What problem does it solve?**
Currently, all tool/function invocations in the Agent Framework are synchronous within the agent's request-response cycle. When the LLM requests a function call, `FunctionInvokingChatClient` immediately invokes it, waits for the result, and feeds it back — all within a single `RunAsync` call. This doesn't work for real-world async operations where a tool triggers an external process (API call, workflow, human task) that takes seconds to hours to complete. The agent should not hold a connection/thread waiting; it should yield control and resume when the result arrives.
The existing `requireApproval` (human-in-the-loop) pattern is close — it suspends the agent via `ExternalInputRequest`/`ExternalInputResponse` — but it gates **before** the tool call (accept/reject). This feature would use the same suspend/resume principle but for the **post-dispatch** phase: the tool call is actually made, then the agent suspends until the result comes back.
**What would the expected behavior be?**
The pattern should mirror exactly how `ApprovalRequiredAIFunction` / `FunctionApprovalRequestContent` works today for HITL, but for async tool dispatch:
**How HITL approval works today (for reference):**
1. Tool is wrapped in `ApprovalRequiredAIFunction`
2. When the LLM requests that tool, `FunctionInvokingChatClient` does NOT execute it — instead it returns immediately with `FunctionApprovalRequestContent` in the response messages
3. The caller parses `response.Messages` for `FunctionApprovalRequestContent` items
4. The caller decides (approve/reject), then calls `approvalRequest.CreateResponse(approved)` to create a `FunctionApprovalResponseContent`
5. The caller sends the response back: `agent.RunAsync([new ChatMessage(ChatRole.Tool, [approvalResponse])], session)`
6. The framework maps the response back by `CallId`, executes (or skips) the tool, and the LLM continues
**Proposed async tool pattern (same shape, different semantics):**
1. Tool is wrapped in `AsyncAIFunction` (analogous to `ApprovalRequiredAIFunction`)
2. When the LLM requests that tool, the framework **dispatches the actual call** (the tool's method runs, sends request to external system, returns a correlation/tracking ID)
3. The agent returns immediately with `AsyncFunctionRequestContent` in the response messages — containing the `CallId`, function name, arguments, and the dispatch result (e.g., tracking ID)
4. The caller parses `response.Messages` for `AsyncFunctionRequestContent` items, stores the pending call info
5. Later, when the external system provides the result, the caller calls `asyncRequest.CreateResponse(resultPayload)` to create an `AsyncFunctionResponseContent`
6. The caller sends the response back: `agent.RunAsync([new ChatMessage(ChatRole.Tool, [asyncResponse])], session)`
7. The framework maps the response back by `CallId`, the async function's registered callback processes the result, and it becomes available to the LLM to continue
**Key difference from HITL:** The tool method **does execute** during step 2 (dispatching the request), but the final result is not yet available. The response message signals "I started this work, here's how to identify it — send me the result when it's done."
```
Caller Agent Framework LLM / External System
| | |
|-- RunAsync(messages) ------------>| |
| |-- LLM requests tool call |
| |-- Detect AsyncAIFunction wrapper |
| |-- Execute tool (dispatch only) -->| (sends request)
| |-- Tool returns tracking ID |
| |-- Return AsyncFunctionRequestContent
|<-- AgentResponse ----------------| |
| { Messages containing | |
| AsyncFunctionRequestContent | |
| (callId, name, trackingId) } | |
| | |
| (caller stores pending call, | |
| time passes...) | |
| | |
| (external system completes) <----|----------------------------------|
| | |
| asyncRequest.CreateResponse(result) |
|-- RunAsync([ChatMessage( | |
| ChatRole.Tool, | |
| [asyncResponse])], session) -->| |
| |-- Map by CallId |
| |-- Process via callback |
| |-- Feed result to LLM |
| |-- LLM continues conversation |
|<-- AgentResponse (final) ---------| |
```
**Are there any alternatives you've considered?**
1. **Using `requireApproval` as a workaround**: Gate the call, have the external handler execute the function and return the result. But this conflates "approval" semantics with "async execution" and requires the external handler to know function implementation details or have workaround logic to pass in approval and internally mask the response as immediatly available.
2. **DurableTask orchestrations**: Works for durable scenarios but requires DurableTask infrastructure. Many deployments need lightweight async tools without orchestration overhead.
3. **Custom middleware via `FunctionInvocationDelegatingAgent`**: Could intercept the call and throw a "suspend" exception, but the framework has no first-class support for capturing and restoring the invocation state. This is the current approach that I am using and it needs some additional management code in the caller code.
## Code Sample
### 1. Defining the async tool (analogous to `ApprovalRequiredAIFunction`)
```csharp
// The tool method dispatches work and returns a tracking/correlation ID.
// It does NOT wait for the final result — that comes later via callback.
[Description("Submits order to fulfillment system")]
static async Task SubmitOrder(string orderId, string[] items)
{
// Sends request to external system, returns immediately with tracking ID
var trackingId = await fulfillmentClient.SubmitAsync(orderId, items);
return trackingId;
}
// Wrap with AsyncAIFunction — just like ApprovalRequiredAIFunction wraps for approval
AIAgent agent = chatClient.AsAIAgent(
instructions: "You are a helpful order assistant",
tools: [new AsyncAIFunction(AIFunctionFactory.Create(SubmitOrder))]);
```
### 2. Caller-side: detect async requests, collect results, re-invoke (mirrors HITL approval loop exactly)
```csharp
// STEP 1: Call agent
AgentSession session = await agent.CreateSessionAsync();
AgentResponse response = await agent.RunAsync("Submit order #123 with items A, B, C", session);
// STEP 2: Detect async function requests (same pattern as FunctionApprovalRequestContent)
List asyncRequests = response.Messages
.SelectMany(m => m.Contents)
.OfType()
.ToList();
// STEP 3: Loop until all async calls resolved (mirrors the approval while-loop)
while (asyncRequests.Count > 0)
{
// BUILD RESPONSE MESSAGES — for each pending async call, wait for the result
List asyncResponses = new();
foreach (var asyncRequest in asyncRequests)
{
Console.WriteLine($"Async tool dispatched: {asyncRequest.FunctionCall.Name}");
Console.WriteLine($" Dispatch result (tracking ID): {asyncRequest.DispatchResult}");
// ... wait for external system to complete (poll, webhook, queue, etc.) ...
string externalResult = await WaitForExternalResult(asyncRequest.DispatchResult);
// Create response — same pattern as approvalRequest.CreateResponse(approved)
asyncResponses.Add(new ChatMessage(ChatRole.Tool,
[asyncRequest.CreateResponse(externalResult)]));
}
// STEP 4: Send responses back to agent (same as sending approval responses)
response = await agent.RunAsync(asyncResponses, session);
// Check for more async requests (agent may chain multiple async tools)
asyncRequests = response.Messages
.SelectMany(m => m.Contents)
.OfType()
.ToList();
}
// Final response — LLM has all results, conversation continues
Console.WriteLine($"\nAgent: {response}");
```
### 3. Streaming variant (mirrors HITL sample)
```csharp
List asyncResponses = [];
do
{
asyncResponses.Clear();
List updates = [];
await foreach (var update in agent.RunStreamingAsync(messages, session))
{
updates.Add(update);
foreach (AIContent content in update.Contents)
{
switch (content)
{
case AsyncFunctionRequestContent asyncRequest:
asyncResponses.Add(asyncRequest);
break;
case TextContent textContent:
Console.Write(textContent.Text);
break;
}
}
}
// Wait for all async results AFTER streaming completes
for (int i = 0; i < asyncResponses.Count; i++)
{
var asyncRequest = (AsyncFunctionRequestContent)asyncResponses[i];
string result = await WaitForExternalResult(asyncRequest.DispatchResult);
asyncResponses[i] = asyncRequest.CreateResponse(result);
}
AgentResponse response = updates.ToAgentResponse();
messages.AddRange(response.Messages);
foreach (AIContent asyncResponse in asyncResponses)
{
messages.Add(new ChatMessage(ChatRole.Tool, [asyncResponse]));
}
}
while (asyncResponses.Count > 0);
```
## Language/SDK
- Both (.NET and Python — code examples above are .NET; the same pattern should apply to the Python SDK)
## Additional Context
**Design follows the established HITL approval pattern.** The proposed implementation mirrors the existing `ApprovalRequiredAIFunction` / `FunctionApprovalRequestContent` / `FunctionApprovalResponseContent` flow exactly:
| HITL Approval (exists today) | Async Tool (proposed) |
|---|---|
| `ApprovalRequiredAIFunction` wrapper | `AsyncAIFunction` wrapper |
| `FunctionApprovalRequestContent` in response | `AsyncFunctionRequestContent` in response |
| `approvalRequest.CreateResponse(approved)` | `asyncRequest.CreateResponse(resultPayload)` |
| `FunctionApprovalResponseContent` sent back | `AsyncFunctionResponseContent` sent back |
| Framework maps by `CallId`, executes/skips tool | Framework maps by `CallId`, processes result via callback |
| Caller loop: parse requests → respond → re-invoke | Caller loop: parse requests → get results → re-invoke |
**Intent to contribute:** I intend to submit a PR implementing this feature. Proposed implementation approach:
1. **New content types** (in `Microsoft.Extensions.AI` or Agent Framework abstractions):
- `AsyncFunctionRequestContent` — like `FunctionApprovalRequestContent`, carries `CallId`, `FunctionCall`, and `DispatchResult` (the tracking/correlation ID from the initial tool dispatch)
- `AsyncFunctionResponseContent` — like `FunctionApprovalResponseContent`, carries `CallId` and the final result payload
- `CreateResponse(result)` method on the request type for easy response construction
2. **`AsyncAIFunction` wrapper** (analogous to `ApprovalRequiredAIFunction`):
- Wraps any `AIFunction`
- When `FunctionInvokingChatClient` invokes it: executes the tool (dispatch), captures the return value as `DispatchResult`, then emits `AsyncFunctionRequestContent` instead of `FunctionResultContent`
- When the caller sends back `AsyncFunctionResponseContent`, the framework maps by `CallId` and makes the final result available to the LLM
3. **Streaming event generators** (for OpenAI Hosting / DevUI):
- `AsyncFunctionRequestEventGenerator` — emits `response.async_function.requested` events (mirrors `response.function_approval.requested`)
- `AsyncFunctionResponseEventGenerator` — emits `response.async_function.responded` events
4. **Declarative workflow support**: `async: true` on `InvokeFunctionTool`, leveraging existing `ExternalInputRequest`/`ExternalInputResponse` plumbing
5. **Sample & tests**: Getting started sample mirroring `Agent_Step04_UsingFunctionToolsWithApprovals`, unit tests for the request/response lifecycle
I'd appreciate feedback on the design direction before starting the PR.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the existing ApprovalRequiredAIFunction, FunctionApprovalRequestContent, and FunctionApprovalResponseContent flow referenced in the issue, including the RunAsync and RunStreamingAsync entry points. Compare the proposed async request/response lifecycle across the .NET and Python SDKs; done means dispatched calls can be represented, resumed by CallId with an external result, and continued through both invocation modes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, python
- Domain
- ai, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100