Feature Request: Using function tools with human in the loop approvals

Feature Request: Tool Call Side Channel and Human-in-the-Loop Approval

Summary

I would like Spring AI's default tool-calling loop to expose two extension points:

1. A tool-call lifecycle listener that can publish model-generated tool_calls and tool execution results to a UI, log, audit system, or other side channel.
2. A pre-execution approval handler that can pause tool execution until a human approves or declines each protected call.

My application currently implements both features by decorating ToolCallingManager. The workaround is functional, but it forces application code to take responsibility for framework-level details such as splitting a batch of tool calls, synthesizing responses for declined calls, and preserving tool-response ordering.

1. Minimal Side-Channel Example

1.1 Why a side channel is needed

A model interaction involving tools looks like this:

Model returns tool_calls
        |
        v
ToolCallingManager executes the tools
        |
        v
Tool results are sent back to the model
        |
        v
Model returns the final text

In the Spring AI 2.0.1 streaming tool loop, the intermediate ChatResponse containing tool_calls is consumed by the advisor and is not forwarded as a normal downstream streaming chunk. As a result, the application normally sees the final text but cannot show which tools the model requested.

The detailed model and tool information I want to expose includes:

• toolCallId
• Tool name
• Model-generated arguments
• Tool execution result

The smallest workaround is to wrap ToolCallingManager. The normal assistant response continues through the original ChatClient stream, while the manager publishes tool details to a separate sink. The two streams are merged only at the SSE boundary.

ChatClient stream: token / reasoning / finish --------+
                                                       +--> SSE
Manager side channel: tool_call / tool_result ---------+

"Side channel" only means observing and forwarding intermediate information. It does not make another model request, and it does not replace Spring AI's normal tool execution.

1.2 Minimal ToolCallingManager decorator

The following example omits event mapping, logging, and defensive checks. The essential behavior is only the two event publications: tool_call before execution and tool_result after execution.

final class ObservableToolCallingManager implements ToolCallingManager {

    private final ToolCallingManager delegate;
    private final Sinks.Many<ChatEvent> events;

    ObservableToolCallingManager(
            ToolCallingManager delegate,
            Sinks.Many<ChatEvent> events) {
        this.delegate = delegate;
        this.events = events;
    }

    @Override
    public List<ToolDefinition> resolveToolDefinitions(
            ToolCallingChatOptions options) {
        return delegate.resolveToolDefinitions(options);
    }

    @Override
    public ToolExecutionResult executeToolCalls(
            Prompt prompt,
            ChatResponse response) {

        events.tryEmitNext(toToolCallEvent(response));

        ToolExecutionResult result =
                delegate.executeToolCalls(prompt, response);

        events.tryEmitNext(toToolResultEvent(result));
        return result;
    }
}

toToolCallEvent extracts id, name, and arguments from the ChatResponse. toToolResultEvent extracts id, name, and responseData from the final ToolResponseMessage in ToolExecutionResult.conversationHistory(). This is only DTO mapping and does not affect the side-channel pattern.

The delegate remains Spring AI's native manager:

ToolCallingManager delegate = ToolCallingManager.builder().build();

The decorator therefore does not change tool selection, argument parsing, execution, or returnDirect behavior. It only adds observability.

1.3 Merging the side channel into SSE

The service creates one sink per request and installs the decorated manager in ToolCallingAdvisor:

Sinks.Many<ChatEvent> toolEvents =
        Sinks.many().unicast().onBackpressureBuffer();

ToolCallingManager manager = new ObservableToolCallingManager(
        ToolCallingManager.builder().build(),
        toolEvents);

ToolCallingAdvisor advisor = ToolCallingAdvisor.builder()
        .toolCallingManager(manager)
        .build();

Flux<ChatEvent> modelEvents = ChatClient.create(chatModel)
        .prompt()
        .user(request.query())
        .advisors(advisor)
        .stream()
        .chatResponse()
        .concatMap(this::toEvents)
        .doFinally(signal -> toolEvents.tryEmitComplete());

return modelEvents.mergeWith(toolEvents.asFlux());

The client can then receive all of the following on the same SSE connection:

data: {"type":"tool_call","toolCalls":[{"id":"call-1","name":"Write","arguments":"{...}"}]}

data: {"type":"tool_result","toolResults":[{"id":"call-1","name":"Write","result":"..."}]}

data: {"type":"token","data":"The file has been written."}

This is the tool-event side channel used by the application: tool details come from the manager, the final answer comes from the model response stream, and both are merged only for transport.

2. Adding Human-in-the-Loop Approval to the Manager

The side channel solves "show the tool call." Human-in-the-loop approval additionally solves "do not execute the tool until a human approves it."

The extended flow is:

1. The manager receives complete tool_calls
2. Publish tool_call through the side channel
3. Publish approval_request for protected tools
4. Wait for POST /chat/approval
5. APPROVE: execute through the native manager
6. DECLINE / timeout / failure: do not execute; synthesize a declined result
7. Publish tool_result through the side channel
8. Spring AI sends the results back to the model and continues generation

2.1 Main-agent manager

The production code keeps the manager as a thin decorator and moves approval behavior into HitlToolCallingGate:

class ObservableToolCallingManager implements ToolCallingManager {

    private final ToolCallingManager delegate;
    private final HitlToolCallingGate gate;

    ObservableToolCallingManager(
            ToolCallingManager delegate,
            Sinks.Many<ChatEvent> sink,
            ToolPolicyProperties policy,
            ApprovalRegistry approvals,
            boolean bypassApproval) {
        this.delegate = Objects.requireNonNull(delegate);
        this.gate = new HitlToolCallingGate(
                sink,
                policy,
                approvals,
                bypassApproval,
                HitlEventFactory.mainAgent());
    }

    @Override
    public List<ToolDefinition> resolveToolDefinitions(
            ToolCallingChatOptions options) {
        return delegate.resolveToolDefinitions(options);
    }

    @Override
    public ToolExecutionResult executeToolCalls(
            Prompt prompt,
            ChatResponse response) {
        return gate.executeToolCalls(prompt, response, delegate);
    }
}

The decorator does not reimplement Spring AI's normal execution path. If no tool call is declined, execution is delegated unchanged.

2.2 Pending approval registry

The SSE request and the approval HTTP request run on different threads. A requestId and CompletableFuture connect the two requests:

@Component
public class ApprovalRegistry {

    public enum Decision { APPROVE, DECLINE }

    private final Map<String, CompletableFuture<Decision>> pending =
            new ConcurrentHashMap<>();   //in-memory registry

    public Pending register(Duration timeout) {
        String requestId = UUID.randomUUID().toString();
        CompletableFuture<Decision> future = new CompletableFuture<>();

        future.orTimeout(timeout.toMillis(), TimeUnit.MILLISECONDS);
        pending.put(requestId, future);
        future.whenComplete((decision, error) -> pending.remove(requestId));

        return new Pending(requestId, future);
    }

    public boolean complete(String requestId, Decision decision) {
        CompletableFuture<Decision> future = pending.get(requestId);
        return future != null && future.complete(decision);
    }

    public record Pending(
            String requestId,
            CompletableFuture<Decision> future) {
    }
}

The registry only correlates the pending stream operation with the later HTTP decision. It does not execute tools. Entries are removed when their future completes, times out, or is cancelled.

This example is intentionally an in-memory rendezvous registry, not a durable repository. A multi-instance or restart-safe implementation should persist approval records through an ApprovalRepository and use a separate notification mechanism to wake the waiting execution. The CompletableFuture itself is process-local and should not be treated as persistent state.

2.3 Core approval gate

The following code retains the important branches from the current implementation and omits event conversion, logging, and argument validation:

final class HitlToolCallingGate {

    private final Sinks.Many<ChatEvent> sink;
    private final ToolPolicyProperties policy;
    private final ApprovalRegistry approvals;
    private final boolean bypassApproval;
    private final HitlEventFactory eventFactory;

    ToolExecutionResult executeToolCalls(
            Prompt prompt,
            ChatResponse response,
            ToolCallingManager delegate) {

        emitToolCall(response);                          // Publish model tool-call details
        Set<String> declined = awaitApprovals(response); // Wait before execution

        ToolExecutionResult result = declined.isEmpty()
                ? delegate.executeToolCalls(prompt, response)
                : executeApprovedOnly(prompt, response, declined, delegate);

        emitToolResult(result);                          // Publish real or synthesized results
        return result;
    }

    private Set<String> awaitApprovals(ChatResponse response) {
        if (bypassApproval) {
            return Set.of();
        }

        List<PendingApproval> pending = new ArrayList<>();
        for (ToolCall call : flattenToolCalls(response)) {
            if (!policy.requiresApproval(call.name())) {
                continue;
            }

            ApprovalRegistry.Pending approval =
                    approvals.register(policy.getTimeout());
            pending.add(new PendingApproval(approval, call));

            sink.tryEmitNext(eventFactory.approvalRequest(
                    approval.requestId(),
                    call.id(),
                    call.name(),
                    call.arguments()));
            sink.tryEmitNext(ChatEvent.heartbeat());
        }

        Set<String> declinedIds = new HashSet<>();
        for (PendingApproval item : pending) {
            if (awaitDecision(item.approval()) != Decision.APPROVE) {
                declinedIds.add(item.toolCall().id());
            }
        }
        return declinedIds;
    }

    private Decision awaitDecision(ApprovalRegistry.Pending pending) {
        try {
            return pending.future().get();
        }
        catch (InterruptedException ex) {
            Thread.currentThread().interrupt();
            return Decision.DECLINE;
        }
        catch (ExecutionException | RuntimeException ex) {
            return Decision.DECLINE;
        }
    }

    private record PendingApproval(
            ApprovalRegistry.Pending approval,
            ToolCall toolCall) {
    }
}

The most important safety rule is that only an explicit APPROVE permits execution. A decline, timeout, interruption, cancellation, or other failure is treated as DECLINE.

An immediate heartbeat is published after each approval request so that buffered SSE data reaches the client promptly. The stream pipeline also publishes periodic heartbeats while the approval remains pending.

2.4 Why a declined call cannot simply return an empty result

The model can request multiple tools in one turn:

call-1: Read   -> No approval required; execute
call-2: Write  -> Declined; do not execute
call-3: Bash   -> Approved; execute

Declining one call must not discard the whole batch. It is also invalid to execute only approved calls and omit declined calls from the response. The next model turn still needs one tool response for every original toolCallId.

The current implementation therefore:

1. Sends only approved calls to the native manager.
2. Synthesizes a ToolResponse for every declined call.
3. Reassembles all responses in the original tool_calls order.

The core implementation is:

private ToolExecutionResult executeApprovedOnly(
        Prompt prompt,
        ChatResponse response,
        Set<String> declinedIds,
        ToolCallingManager delegate) {

    AssistantMessage assistant = response.getResults().stream()
            .map(Generation::getOutput)
            .filter(message -> !message.getToolCalls().isEmpty())
            .findFirst()
            .orElseThrow();

    List<ToolCall> allCalls = assistant.getToolCalls();
    List<ToolCall> approvedCalls = allCalls.stream()
            .filter(call -> !declinedIds.contains(call.id()))
            .toList();

    Map<String, ToolResponseMessage.ToolResponse> responsesById =
            new LinkedHashMap<>();

    if (!approvedCalls.isEmpty()) {
        AssistantMessage approvedAssistant = AssistantMessage.builder()
                .content(assistant.getText())
                .properties(assistant.getMetadata())
                .toolCalls(approvedCalls)
                .build();

        ToolExecutionResult approvedResult = delegate.executeToolCalls(
                prompt,
                new ChatResponse(List.of(new Generation(approvedAssistant))));

        ToolResponseMessage executed =
                (ToolResponseMessage) approvedResult.conversationHistory().getLast();
        executed.getResponses().forEach(
                item -> responsesById.put(item.id(), item));
    }

    for (ToolCall call : allCalls) {
        if (declinedIds.contains(call.id())) {
            responsesById.put(call.id(), new ToolResponseMessage.ToolResponse(
                    call.id(),
                    call.name(),
                    "The user declined this tool. It was not executed."));
        }
    }

    List<ToolResponseMessage.ToolResponse> orderedResponses = allCalls.stream()
            .map(call -> responsesById.get(call.id()))
            .toList();

    List<Message> history = new ArrayList<>(prompt.getInstructions());
    history.add(assistant);
    history.add(ToolResponseMessage.builder()
            .responses(orderedResponses)
            .build());

    return ToolExecutionResult.builder()
            .conversationHistory(history)
            .build();
}

The ordering cannot be changed. Some model providers strictly validate the relationship between the assistant's tool_calls and the subsequent tool messages. Incorrect ordering causes the next model request to fail.

2.5 Service wiring

The application installs this manager only for streaming main-agent requests:

Sinks.Many<ChatEvent> toolEventSink =
        Sinks.many().unicast().onBackpressureBuffer();

ToolCallingManager manager = new ObservableToolCallingManager(
        ToolCallingManager.builder().build(),
        toolEventSink,
        toolPolicy,
        approvalRegistry,
        Boolean.TRUE.equals(request.bypassApproval()));

ToolCallingAdvisor advisor = ToolCallingAdvisor.builder()
        .toolCallingManager(manager)
        .build();

Flux<ChatEvent> main = spec.stream()
        .chatResponse()
        .concatMap(this::toEvents)
        .doOnComplete(toolEventSink::tryEmitComplete)
        .doOnError(error -> toolEventSink.tryEmitComplete());

return main
        .mergeWith(toolEventSink.asFlux())
        .doFinally(signal -> toolEventSink.tryEmitComplete());

Protected tools are configured by name:

chat:
  hitl:
    required-tools: [Write, Edit, Bash]
    timeout: 3m
    heartbeat-interval: 15s

2.6 Approval callback controller

The approval_request event includes a requestId. After the user approves or declines the call, the client submits the decision through a separate HTTP endpoint:

@RestController
public class ChatController {

    private final ChatService chatService;
    private final ApprovalRegistry approvalRegistry;

    @PostMapping(
            value = "/chat/stream",
            produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public Flux<ChatEvent> streamChat(
            @RequestBody ChatRequest request,
            @RequestHeader HttpHeaders headers,
            HttpServletResponse response) {
        response.setHeader("X-Accel-Buffering", "no");
        return chatService.streamChat(
                ChatRequestContext.of(request, headers));
    }

    @PostMapping("/chat/approval")
    public ApprovalResponse approve(@RequestBody ApprovalRequest body) {
        if (body == null || body.requestId() == null
                || body.requestId().isBlank()) {
            return ApprovalResponse.miss();
        }

        Decision decision = body.isApprove()
                ? Decision.APPROVE
                : Decision.DECLINE;

        boolean accepted = approvalRegistry.complete(
                body.requestId(), decision);
        return accepted
                ? ApprovalResponse.ok()
                : ApprovalResponse.miss();
    }
}

Only an explicit approve value is treated as approval. Every other value is treated as a decline. An unknown or expired requestId returns accepted=false and never authorizes execution.

A complete interaction looks like this:

POST /chat/stream
  <- tool_call(call-1, Write, arguments)
  <- approval_request(request-9, call-1, Write, arguments)
  <- heartbeat

POST /chat/approval
  -> {"requestId":"request-9","decision":"approve"}
  <- {"accepted":true}

Original SSE stream continues
  <- tool_result(call-1, Write, result)
  <- token(...)
  <- finish

3. Human-in-the-Loop for Subagents

3.1 Why a subagent also needs its own manager

After the main agent calls Task, the subagent creates its own ChatClient and ToolCallingAdvisor. The main agent's manager can observe the outer Task call, but it cannot observe internal Read, Write, or Bash calls made by the subagent.

The subagent must therefore install a manager in its own tool loop:

Main-agent ChatClient
  |
  +-- Task tool
        |
        v
SandboxSubagentExecutor
        |
        v
Subagent ChatClient
        |
        v
SubagentToolCallingManager
        |
        +-- Reuses HitlToolCallingGate
        +-- Reuses ApprovalRegistry
        +-- Publishes to the main request's SSE sink

This is not a second HITL implementation. The main agent and subagent share the same components and rules:

Concern	Main agent	Subagent
Approval policy	chat.hitl.required-tools	Same configuration
Approval registry	ApprovalRegistry	Same singleton
Callback endpoint	POST /chat/approval	Same endpoint
Per-request bypass	bypassApproval	Inherited from the main request
Approval gate	HitlToolCallingGate	Same implementation
SSE events	tool_call, etc.	subagent_tool_call, etc.

3.2 Subagent manager

SubagentToolCallingManager has the same structure as the main-agent decorator. It still delegates real tool execution to Spring AI's native manager, but selects a subagent-specific event factory when it creates the gate:

class SubagentToolCallingManager implements ToolCallingManager {

    private final ToolCallingManager delegate;
    private final HitlToolCallingGate gate;

    SubagentToolCallingManager(
            ToolCallingManager delegate,
            Sinks.Many<ChatEvent> sink,
            String subagentName,
            ToolPolicyProperties policy,
            ApprovalRegistry approvals,
            boolean bypassApproval) {
        this.delegate = Objects.requireNonNull(delegate);
        this.gate = new HitlToolCallingGate(
                sink,
                policy,
                approvals,
                bypassApproval,
                HitlEventFactory.subagent(subagentName));
    }

    @Override
    public List<ToolDefinition> resolveToolDefinitions(
            ToolCallingChatOptions options) {
        return delegate.resolveToolDefinitions(options);
    }

    @Override
    public ToolExecutionResult executeToolCalls(
            Prompt prompt,
            ChatResponse response) {
        return gate.executeToolCalls(prompt, response, delegate);
    }
}

Approval, timeout handling, partial declines, and response ordering are all handled by the previously described HitlToolCallingGate. There is no duplicate subagent approval algorithm.

3.3 Distinguishing the event source

The shared gate needs to tell the client whether an event came from the main agent or a named subagent. The current implementation isolates that difference in HitlEventFactory:

interface HitlEventFactory {

    ChatEvent toolCall(List<ChatEvent.ToolCallRef> calls);

    ChatEvent approvalRequest(
            String requestId,
            String toolCallId,
            String toolName,
            String arguments);

    ChatEvent toolResult(List<ChatEvent.ToolResultRef> results);

    static HitlEventFactory mainAgent() {
        return MainAgent.INSTANCE;
    }

    static HitlEventFactory subagent(String name) {
        return new Subagent(name);
    }
}

The subagent factory only changes the event type and adds the subagent name:

record Subagent(String name) implements HitlEventFactory {

    @Override
    public ChatEvent toolCall(List<ChatEvent.ToolCallRef> calls) {
        return ChatEvent.subagentToolCall(name, calls);
    }

    @Override
    public ChatEvent approvalRequest(
            String requestId,
            String toolCallId,
            String toolName,
            String arguments) {
        return ChatEvent.subagentApprovalRequest(
                name, requestId, toolCallId, toolName, arguments);
    }

    @Override
    public ChatEvent toolResult(List<ChatEvent.ToolResultRef> results) {
        return ChatEvent.subagentToolResult(name, results);
    }
}

The event mapping is straightforward:

Main-agent event	Subagent event	Additional subagent field
tool_call	subagent_tool_call	name
approval_request	subagent_approval_request	name
tool_result	subagent_tool_result	name

Heartbeats remain plain heartbeat events because they keep the entire SSE connection alive and are not business output from a particular agent.

3.4 Passing shared dependencies from ChatService

When the main request creates SandboxSubagentExecutor, it passes the current stream sink, approval policy, registry, and bypass flag:

SandboxSubagentExecutor executor = new SandboxSubagentExecutor(
        Map.of("default", modelRouter.chatClientBuilder(req.modelName())),
        List.copyOf(executorTools),
        skillDirs,
        streamSink,
        () -> buildOptionsBuilder(req, provider),
        toolPolicy,
        approvalRegistry,
        Boolean.TRUE.equals(req.bypassApproval()),
        agentContext == null ? "" : agentContext.promptBlock());

streamSink is the side-channel sink already used by the main /chat/stream request. The subagent does not need a second SSE connection.

3.5 Installing the manager in the subagent ChatClient

SandboxSubagentExecutor installs the manager when it builds each subagent ChatClient:

private ChatClient createTaskChatClient(
        ClaudeSubagentDefinition claudeSubagent) {
    ChatClient.Builder builder =
            doFindChatClientBuilder(claudeSubagent).clone();

    // Filter and register subagent tools using tools/disallowedTools first.

    ToolCallingAdvisor.Builder advisor = ToolCallingAdvisor.builder();
    if (sink != null) {
        ToolCallingManager manager = new SubagentToolCallingManager(
                ToolCallingManager.builder().build(),
                sink,
                claudeSubagent.getName(),
                toolPolicy,
                approvalRegistry,
                bypassApproval);
        advisor.toolCallingManager(manager);
    }

    builder.defaultAdvisors(advisor.build());
    return builder.build();
}

sink != null means this is a streaming /chat/stream request. The non-streaming /chat path does not install the manager and therefore does not enter HITL, matching the current main-agent behavior.

The subagent's frontmatter tools and disallowedTools settings filter the available tool set first. Approval is requested only for a tool that the subagent can actually call and that also matches required-tools.

3.6 Subagent approval interaction

Subagent requestId values are still globally unique UUIDs created by the shared ApprovalRegistry. The controller does not need to know whether an approval originated from the main agent or a subagent:

Main SSE stream
  <- tool_call(Task)
  <- subagent_start(name=researcher)
  <- subagent_tool_call(name=researcher, call-2, Bash, arguments)
  <- subagent_approval_request(
         name=researcher,
         requestId=request-10,
         toolCallId=call-2)
  <- heartbeat

POST /chat/approval
  -> {"requestId":"request-10","decision":"approve"}
  <- {"accepted":true}

Original SSE stream continues
  <- subagent_tool_result(name=researcher, call-2, Bash, result)
  <- subagent_token(name=researcher, ...)
  <- subagent_finish(name=researcher)
  <- tool_result(Task, ...)
  <- token(...)
  <- finish

The subagent tool is not executed while approval is pending. A decline, timeout, interruption, or unknown requestId follows the same fail-safe behavior and is treated as a decline.

4. Proposed Spring AI Extension Points

ToolCallingManager proves that these features are possible, but it is too coarse-grained for this use case. To add observability and approval, application code must understand and duplicate part of the default tool-loop protocol.

Spring AI should provide at least the following three extension points or framework responsibilities.

4.1 Tool-call lifecycle listener

The listener should observe execution without changing its result:

public interface ToolCallListener {

    default void beforeToolExecution(ToolCallBatch batch) {
    }

    default void afterToolExecution(
            ToolCallBatch batch,
            ToolExecutionResult result) {
    }
}

It should receive complete, normalized tool call IDs, names, and arguments. Applications could then implement SSE side channels, auditing, metrics, and debugging UIs without replacing the whole manager.

Listener failures should be isolated from tool execution by default.

4.2 Pre-execution approval handler

The approval handler should decide whether each call may execute:

public interface ToolApprovalHandler {

    CompletionStage<Map<String, ToolApprovalDecision>> approve(
            ToolCallBatch batch,
            ToolApprovalContext context);
}

public enum ToolApprovalDecision {
    APPROVE,
    APPROVE_AND_MODIFY,
    DECLINE
}

Decisions should be keyed by toolCallId, rather than represented by a single boolean for the whole batch, because only some calls in a turn may require approval.

ToolApprovalContext should provide at least a conversation ID, tool context, optional source metadata such as an agent name, and timeout or cancellation signals. This would let applications correlate calls with an external approval system.

4.3 Protocol correctness should remain framework-owned

After receiving approval decisions, DefaultToolCallingManager should be responsible for:

• Executing only approved calls.
• Producing a standard ToolResponse for declined calls.
• Preserving the original tool_calls order.
• Correctly handling returnDirect and tool-call limits.
• Defaulting to decline on timeout, cancellation, or failure.

This is the most important ownership boundary. The application should supply the decisions; it should not have to reconstruct assistant and tool message history.

4.4 Suggested configuration API

The user-facing setup could remain small:

ToolCallingManager manager = ToolCallingManager.builder()
        .toolCallListener(toolCallListener)
        .toolApprovalHandler(toolApprovalHandler)
        .approvalTimeout(Duration.ofMinutes(3))
        .build();

For an initial implementation, the framework could wait for the CompletionStage on its dedicated tool-execution scheduler. Durable checkpoint and resume APIs could be added later; they do not need to block this minimal feature.

Start by tracing the ToolCallingManager and ToolCallingAdvisor entry points described in the issue, including how ChatResponse tool calls become ToolExecutionResult responses. Define the lifecycle-listener and pre-execution approval extension points, then verify batch ordering, declined-call responses, timeouts, and streaming behavior with focused tests. Done means side-channel events and explicit human approval work without changing native execution when approval is not required.