cloudflare / cloudflare/agents

Support client-side tools in headless AIChatAgent agent-tool runs

Open
#1,420 0 comments 0 reactions 1 assignee Claimed by @ben-reitz View on GitHub
enhancement
Dominant language
TypeScript
Stars
5.6k
Forks
711
Avg merge
1d 20h
Merged PRs (30d)
53

Description

## Summary

AIChatAgent subclasses can now be invoked headlessly as agent-tool children via `runAgentTool()` / `agentTool()`. Server-side AI SDK tools work in that mode, but browser-provided client tools do not, because the child turn runs without a browser WebSocket connection capable of executing `onToolCall` and returning tool outputs.

This issue tracks a follow-up design for supporting, or intentionally mediating, client-side tools during headless AIChatAgent agent-tool runs.

## Current behavior

When an `AIChatAgent` is used as an agent-tool child:

- The parent calls `runAgentTool(ChildAgent, { input, runId, ... })`.
- The child starts a headless turn through `AIChatAgent.saveMessages()`.
- The child receives a synthetic user message from `formatAgentToolInput()`.
- The child streams `CF_AGENT_USE_CHAT_RESPONSE` chunks back to the parent as `agent-tool-event` frames.
- The parent records the run in `cf_agent_tool_runs`; the child records its own run/request mapping in `cf_ai_chat_agent_tool_runs`.

This works for normal model output and server-side tools, because everything needed to complete the turn lives in the child Durable Object.

It does not work for client-side tools, because AIChatAgent client tools are normally supplied by a browser chat request and completed by the browser through `useAgentChat({ onToolCall })`. A headless agent-tool run has no active child browser connection, no child `onToolCall` handler, and no browser round trip to provide tool results.

## Why this is not a Think problem

Think's tool surface is server-owned. Tools come from `getTools()` / server-side AI SDK tool definitions, and tool execution happens inside the Durable Object turn. A Think agent-tool child can therefore complete headlessly because it does not depend on a browser to execute missing tool calls.

AIChatAgent supports a separate browser-provided tool path:

- The client can send `clientTools` schemas in the chat request.
- The server can expose those schemas to the model via `createToolsFromClientSchemas()`.
- When the model calls one of those tools, the browser-side `onToolCall` handler is expected to execute it and send the result back.

That coupling to a live browser chat connection is what breaks in a headless child run.

## Desired outcome

Design and implement a clear story for AIChatAgent agent-tool runs that need client-side tools. Possible directions include:

1. **Parent-mediated client tool bridge**
- Child emits a client-tool request as an agent-tool event to the parent.
- Parent forwards it over the existing parent browser connection.
- Parent client executes `onToolCall` and sends the result back to the parent.
- Parent resumes/continues the child run with the tool output.

2. **Explicit unsupported mode with better diagnostics**
- Detect when a headless child turn attempts to use browser-provided tools.
- Fail with a structured error explaining that client tools are unavailable in agent-tool runs.
- Point users toward server-side tools or a future bridge API.

3. **Server-side declaration/override path**
- Let AIChatAgent children provide server equivalents for tools that are browser-provided in normal chat.
- This may be enough for many apps where the child can use Durable Object state, parent RPC, or external APIs instead of browser APIs.

The right solution may combine (2) for the short term and (1)/(3) as a richer API.

## Constraints and design notes

- Do not silently hang the child turn waiting for a browser tool result that can never arrive.
- Preserve the existing agent-tool event stream and retained run model.
- Avoid serializing non-transferable browser callbacks through Durable Object RPC.
- Consider replay/resume semantics: a client-tool request and result may need durable representation so refreshes do not lose the pending state.
- Consider cancellation: aborting the parent run should cancel any pending client-tool request and unblock the child.
- Consider security: the parent should decide which browser/client tools a child may request, and guessed child run IDs must not grant tool access.
- Keep server-side AI SDK tools working unchanged.

## Relevant files / APIs

- `packages/ai-chat/src/index.ts`
- `AIChatAgent.saveMessages()`
- `AIChatAgent.startAgentToolRun()`
- `AIChatAgent.formatAgentToolInput()`
- `AIChatAgent.getAgentToolOutput()` / `getAgentToolSummary()`
- request context fields `_lastClientTools` and `_lastBody`
- `packages/agents/src/index.ts`
- `Agent.runAgentTool()`
- agent-tool event forwarding and parent registry (`cf_agent_tool_runs`)
- `packages/agents/src/agent-tool-types.ts`
- `AgentToolChildAdapter`
- `AgentToolEventMessage`
- `packages/ai-chat/src/tests/agent-tools.test.ts`
- current AIChatAgent agent-tool coverage for server/headless behavior
- `docs/agent-tools.md`
- current docs explicitly state that AIChatAgent agent-tool turns are headless and should use server-side tools

## Acceptance criteria

- AIChatAgent agent-tool runs that require client-side tools either complete through a documented bridge or fail quickly with a structured, actionable error.
- The behavior is covered by worker tests, including replay/refresh, cancellation, and missing-client cases.
- Docs explain when to use server-side tools, when browser tools are unsupported, and how to opt into any bridge API.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.