microsoft / microsoft/agent-framework

Python: [Feature]: Let agents (not just workflows) raise a generic native AG-UI interrupt for custom-UI human input

Open
#6,975 2 comments 0 reactions 1 assignee View on GitHub

Nobody has claimed this yet.

ag-ui harness python
Dominant language
Python
Stars
13.6k
Forks
2.3k
Avg merge
2d 45m
Merged PRs (30d)
358

Description

### Description

## Ask

An "ask the user clarifying questions" tool. The model calls `ask_user_questions({ questions: [...] })`; the run should **pause**; the frontend should render a **custom card** (clickable options, multi-select, and a free-text "Other" field — not an approve/reject dialog); the user's selections/typed text should come back as the **tool result** so the model continues with the answers.

More generally, any human-input HITL that is *not* a boolean approval: free-form text prompts, structured forms, pickers, disambiguation menus.

The client side is already ready for this: CopilotKit's `useInterrupt` registers `availableInterrupts` and resumes with `resolve(payload)`, and can render arbitrary UI for an interrupt `value`. Only the **server-side emission** on the agent path is missing.

## Why the existing mechanisms don't cover it

1. **Function-approval interrupt** (`_agent_run.py`, `require_confirmation`) — the only interrupt the agent runner emits:
```python
# agent_framework_ag_ui/_agent_run.py:1252
flow.interrupts.append({
"id": str(confirm_id),
"value": {
"type": "function_approval_request",
"function_call": {"call_id": tool_call_id, "name": tool_name, "arguments": function_arguments},
},
})
```
It is boolean-shaped (accept/reject) and **re-executes the tool** after approval — so a free-form answer isn't a first-class return value; it has to be smuggled through the approval-response payload and read back out of band. It is also keyed to a synthetic `confirm_id`, not the original tool call id, and renders as the approval dialog unless the app special-cases the tool name.

2. **Client-declared frontend tool** (`useFrontendTool` / `useHumanInTheLoop` with `parameters`) — the natural "frontend tool" shape, but the CopilotKit runtime **does not forward that tool's result to the AG-UI backend**: the backend receives the assistant tool call with **no** following `tool` message, so the model never sees the answer and re-asks. (Reproduced against CopilotKit 1.61.x and 1.62.x.) So a purely client-side frontend tool cannot round-trip data to an AG-UI agent.

3. **Workflow `request_info`** — the *only* place a generic native interrupt is produced today:
```python
# agent_framework_ag_ui/_workflow_run.py:87
def _interrupt_entry_for_request_event(request_event):
return {"id": str(request_id), "value": } # arbitrary value, keyed by request_id
```
built from `WorkflowContext.request_info(request_data, response_type)` and resumed into an executor's `@response_handler`. This is exactly the generic mechanism we want — but it requires adopting the **workflow** programming model. An application built on `create_harness_agent` (agent path) cannot use it without restructuring the agent into a workflow.

So the generic resume half is present and agent-agnostic (`_resume_to_tool_messages` already keys the resume value to the interrupt id); only a generic **emission** on the agent path is missing.

## Proposed feature

Any of the following (ordered by preference); all reuse the existing native interrupt/resume wire protocol and the existing `_resume_to_tool_messages` resume path.

### Option A — a tool-callable "request user input" primitive on the agent path

Let a tool (or middleware) request input and have the agent runner emit a generic interrupt **keyed to that tool call id**, then return the resume value as the tool result (no re-execution):

```python
from agent_framework import RequestUserInput # new

@tool
async def ask_user_questions(questions: list[Question], ctx: ToolContext) -> Answers:
# Pauses the run, emits a native AG-UI interrupt {id: , value: {...}},
# and returns the client's resume payload when the run resumes.
return await ctx.request_user_input(value={"type": "ask_user_questions", "questions": questions},
response_type=Answers)
```

Mirror `_interrupt_entry_for_request_event` on the agent path: emit `flow.interrupts.append({"id": tool_call_id, "value": value})`, and let the existing `_resume_to_tool_messages` deliver the resume value as the `tool` message for `tool_call_id`.

### Option B — a generic interrupt content type distinct from approvals

Introduce a first-class content type (e.g. `UserInputRequestContent` / `HumanInputRequestContent`) that a tool/middleware can yield, which the AG-UI agent runner surfaces as a native interrupt (value = the content's payload) and whose resume value becomes the tool result. Parallel to `FunctionApprovalRequestContent`, but for arbitrary request/response rather than accept/reject.

### Option C — extend approval to carry a typed request + response and skip re-execution

Least invasive but overloads "approval": allow `require_confirmation` interrupts to carry an arbitrary `request`/`response` schema and, when a response *value* (not just `accepted`) is supplied, use it as the tool result instead of re-invoking the tool. Keyed to the tool call id.

In all three, the frontend contract is the already-shipped native interrupt: the client sees `availableInterrupts` + a `{ id, value }` interrupt, renders whatever UI it wants for `value`, and `resolve(payload)`s; the resume payload becomes the tool result.

## Evidence / references (against `agent-framework-ag-ui 1.0.0rc7`)

| What | Location |
|---|---|
| Generic resume: resume value → `tool` message keyed by interrupt id | `agent_framework_ag_ui/_agent_run.py:177` (`_resume_to_tool_messages`), consumed at `:924` |
| `availableInterrupts` / `resume` on the input | `agent_framework_ag_ui/_types.py:86`, `_http_service.py:69,116`, `_client.py:439` |
| Agent runner emits **only** the approval interrupt | `agent_framework_ag_ui/_agent_run.py:1252` (single `flow.interrupts.append`), surfaced at `:1310,1312` |
| Generic interrupt built **only** in the workflow runner from `request_info` | `agent_framework_ag_ui/_workflow_run.py:87` (`_interrupt_entry_for_request_event`), `:100`, `:154` |
| Workflow `request_info` primitive | `agent_framework/_workflows/_workflow_context.py:393`, `_functional.py:193` (raises `WorkflowInterrupted`) |

### Code Sample

```python
from agent_framework import RequestUserInput # new

@tool
async def ask_user_questions(questions: list[Question], ctx: ToolContext) -> Answers:
# Pauses the run, emits a native AG-UI interrupt {id: , value: {...}},
# and returns the client's resume payload when the run resumes.
return await ctx.request_user_input(value={"type": "ask_user_questions", "questions": questions},
response_type=Answers)
```

### Language/SDK

Both

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.