Provide a Tool Execution Callback API After `streamToolCallResponses` Removal

Problem Statement

In Spring AI 2.0 GA, there is no lightweight way to observe individual tool executions from the application layer during streaming chat. This forces users to replace the entire ToolCallingManager just to add before/after hooks.

Background: What Changed from RC1 to GA

RC1 had streamToolCallResponses(true) on ToolCallingAdvisor, which allowed tool-call frames to pass through the downstream Flux<ChatResponse>. GA removed this option entirely (see upgrade notes and spring-ai#6340) and hard-coded a filter in ToolCallingAdvisor.streamWithToolCallResponses():

.filter(ccr -> !this.toolExecutionEligibilityChecker.isToolCallResponse(ccr.chatResponse()));

The removal was correct — tool-call frames in the main Flux could corrupt conversation history when MessageChatMemoryAdvisor recorded unpaired tool-call messages. However, no alternative was provided for users who need to observe tool calls in real time (e.g., sending SSE events to a frontend).

What Users Actually Need

A streaming chat frontend needs to show:

1. When the model decides to call a tool — tool name, call ID, arguments
2. When the tool finishes executing — tool name, result

Without these events, the frontend sees: "text output" → (long pause) → "more text". The tool execution phase is a black box.

Current Workaround: Replace the Entire ToolCallingManager

The only way to achieve this today is to decorate DefaultToolCallingManager and inject a side-channel Sinks.Many:

// User must implement this entire decorator (~80 lines)
class ObservableToolCallingManager implements ToolCallingManager {
    private final ToolCallingManager delegate;
    private final Sinks.Many<ChatEvent> sink;

    @Override
    public ToolExecutionResult executeToolCalls(Prompt prompt, ChatResponse chatResponse) {
        // 1. Extract tool calls from chatResponse, emit "tool_call" to sink
        sink.tryEmitNext(ChatEvent.toolCall(extractToolCalls(chatResponse)));

        // 2. Delegate to DefaultToolCallingManager
        ToolExecutionResult result = delegate.executeToolCalls(prompt, chatResponse);

        // 3. Extract results, emit "tool_result" to sink
        sink.tryEmitNext(ChatEvent.toolResult(extractResults(result)));

        return result;
    }
}

Then merge the side-channel into the main SSE stream:

Sinks.Many<ChatEvent> toolEventSink = Sinks.many().unicast().onBackpressureBuffer();
ToolCallingManager mgr = new ObservableToolCallingManager(
    ToolCallingManager.builder().build(), toolEventSink);

return spec.stream().chatResponse()
    .concatMap(this::toEvents)
    .mergeWith(toolEventSink.asFlux());  

Expected User-Facing Effect: Real-Time Tool Call Visibility in SSE

This section shows what a frontend chat UI should receive via SSE during a streaming conversation with tool calls. Currently this requires the custom ToolCallingManager decorator workaround described above.

SSE Event Types

type	Description
reasoning	Model thinking/reasoning content, streamed token-by-token (DeepSeek R1, Anthropic thinking)
token	Model output text, streamed token-by-token
tool_call	Model decided to call a tool — carries id/name/arguments
tool_result	Tool finished executing — carries id/name/result
finish	End of one LLM round — reason indicates TOOL_CALLS, STOP, or returnDirect

The tool_call / tool_result events can optionally carry a source field to distinguish the tool origin — builtin-tools, skill, mcp, or customer-json-schemal. This allows the frontend to render different icons, labels, or colors per source (e.g., 🧰 builtin, 📦 skill, 🔌 MCP,).

Scenario: Reasoning → Tool Call → FinalAnswer (Most Complex Path)

DeepSeek R1 executes a Skill, calls an API, and delivers JSON via FinalAnswer(returnDirect=true).
reasoning events are truncated for readability — real streams may have hundreds:

data:{"type":"reasoning","data":" user"}
data:{"type":"reasoning","data":" wants"}
data:{"type":"reasoning","data":" me"}
data:{"type":"reasoning","data":" to"}
data:{"type":"reasoning","data":" use"}
data:{"type":"reasoning","data":" Final"}
data:{"type":"reasoning","data":"Answer"}
  ... (hundreds of reasoning tokens as the model plans the Skill execution) ...
data:{"type":"reasoning","data":" and use"}
data:{"type":"reasoning","data":" Final"}
data:{"type":"reasoning","data":"Answer"}
data:{"type":"reasoning","data":".\n\n"}

data:{"type":"tool_call","toolCalls":[{"id":"call_00_ZWrb","name":"Bash","arguments":"{\"command\": \"python3 -c \\\"...\\\"\", \"description\": \"Request API and print raw JSON\"}"}]}
data:{"type":"finish","reason":"TOOL_CALLS","usage":{"promptTokens":6248,"completionTokens":700,"totalTokens":6948}}
data:{"type":"tool_result","toolResults":[{"id":"call_00_ZWrb","name":"Bash","result":"{\"code\":1,\"message\":\"success\",\"result\":{\"loginDenied\":false}}"}]}

data:{"type":"reasoning","data":"The"}
data:{"type":"reasoning","data":" user"}
data:{"type":"reasoning","data":" wants"}
  ... (model decides to use FinalAnswer) ...
data:{"type":"reasoning","data":"Final"}
data:{"type":"reasoning","data":"Answer"}
data:{"type":"reasoning","data":"."}

data:{"type":"tool_call","toolCalls":[{"id":"call_00_jU6w","name":"FinalAnswer","arguments":"{\"payload\": \"{\\\"code\\\":1,\\\"message\\\":\\\"success\\\"}\"}"}]}
data:{"type":"finish","reason":"TOOL_CALLS","usage":{"promptTokens":6524,"completionTokens":110,"totalTokens":6634}}
data:{"type":"tool_result","toolResults":[{"id":"call_00_jU6w","name":"FinalAnswer","result":"{\"code\":1,\"message\":\"success\"}"}]}

data:{"type":"token","data":"{\"code\":1,\"message\":\"success\"}"}
data:{"type":"finish","reason":"returnDirect","usage":{"promptTokens":6524,"completionTokens":110,"totalTokens":6634}}

Timeline per tool-call round:

  reasoning ... (main Flux — model thinking)
→ tool_call     (listener fires onToolExecutionStart)
→ finish TOOL_CALLS (main Flux — this LLM round ended with tool request)
→ tool_result   (listener fires onToolExecutionSuccess)
→ next round or stream ends

Simpler Scenario: One Round of Tool Calls (No Reasoning)

data:{"type":"token","data":"Let me query the database."}
data:{"type":"token","data":"Looking up user information..."}
data:{"type":"tool_call","toolCalls":[{"id":"call_1","name":"queryDatabase","arguments":"{\"sql\":\"SELECT * FROM users\"}"}]}
data:{"type":"finish","reason":"TOOL_CALLS","usage":{"promptTokens":120,"completionTokens":30,"totalTokens":150}}
data:{"type":"tool_result","toolResults":[{"id":"call_1","name":"queryDatabase","result":"[{\"id\":1,\"name\":\"Alice\"}]"}]}
data:{"type":"token","data":"Found user Alice (ID: 1)."}
data:{"type":"finish","reason":"STOP","usage":{"promptTokens":180,"completionTokens":45,"totalTokens":225}}

Why This Matters for Frontend UX

Without tool_call / tool_result events, the frontend sees:

"Let me query the database. Looking up user information..."
  ← (blank — long pause, user thinks it's broken) →
"Found user Alice (ID: 1)."

With these events, the frontend can render:

"Let me query the database. Looking up user information..."
  🔧 Calling: queryDatabase (SELECT * FROM users) ...
  ✅ queryDatabase returned: [{"id":1,"name":"Alice"}]
"Found user Alice (ID: 1)."

The tool_call and tool_result events are exactly what ToolExecutionListener.onToolExecutionStart / onToolExecutionSuccess would produce — without the current workaround of replacing the entire ToolCallingManager.

Optional Enhancement: StreamToolCallObserver for ToolCallingAdvisor

If we also want to cover the case where users need per-round (not per-tool) events at the advisor level, a companion interface could be added to ToolCallingAdvisor.Builder:

public interface StreamToolCallObserver {
    void onToolCallRoundStarted(List<ToolCallRef> calls);
    void onToolCallRoundCompleted(List<ToolResultRef> results);

    record ToolCallRef(String id, String name, String arguments) {}
    record ToolResultRef(String id, String name, String result) {}
}

This is called once per tool-call round (which may contain multiple tools), giving the advisor-level view. It complements ToolExecutionListener which fires per individual tool.

Proposal: ToolExecutionListener — Application-Level Callbacks

Add a lightweight listener interface that users can register via the ToolCallingManager.Builder or Spring auto-discovery.

The Interface

package org.springframework.ai.model.tool;


public interface ToolExecutionListener {

    /**
     * Called before a tool is executed.
     * @param toolCallId the ID of the tool call (from the model)
     * @param toolName the name of the tool
     * @param toolArguments the arguments passed to the tool (JSON string)
     */
    default void onToolExecutionStart(String toolCallId, String toolName,
                                       String toolArguments) {}

    /**
     * Called after a tool executes successfully.
     * @param toolCallId the ID of the tool call
     * @param toolName the name of the tool
     * @param toolResult the result returned by the tool (may be null)
     */
    default void onToolExecutionSuccess(String toolCallId, String toolName,
                                         @Nullable String toolResult) {}

    /**
     * Called when a tool execution throws an exception.
     * @param toolCallId the ID of the tool call
     * @param toolName the name of the tool
     * @param error the exception thrown
     */
    default void onToolExecutionError(String toolCallId, String toolName,
                                       Throwable error) {}
}

Start by reading ToolCallingManager, DefaultToolCallingManager, and ToolCallingAdvisor, focusing on the existing tool execution flow and builder configuration. Define the application-level ToolExecutionListener behavior described in the issue, including start, success, and error callbacks, and verify that applications can observe executions without replacing the entire manager.