spring-projects / spring-projects/spring-ai

How to reject the tool calling while using `User-Controlled Tool Execution`

Open
#4,871 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

How to reject certain tool calling requests in the process of using User-Controlled Tool Execution.

import lombok.RequiredArgsConstructor;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.messages.AssistantMessage;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.model.Generation;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.model.tool.ToolCallingChatOptions;
import org.springframework.ai.model.tool.ToolCallingManager;
import org.springframework.ai.model.tool.ToolExecutionResult;
import org.springframework.ai.tool.ToolCallbackProvider;
import org.springframework.stereotype.Component;

import java.util.Scanner;

@Component
@RequiredArgsConstructor
public class ToolCallingAgent {

    private final ChatClient.Builder builder;
    private final ToolCallbackProvider mcpTools;

    public void run() {
        ToolCallingManager toolCallingManager = ToolCallingManager.builder().build();
        ToolCallingChatOptions chatOptions = ToolCallingChatOptions.builder()
                .temperature(0.1)
                .toolCallbacks(mcpTools.getToolCallbacks())
                .internalToolExecutionEnabled(false)
                .build();
        Prompt prompt = new Prompt("""
                Execute tool calls when necessary. If a response does not contain tool call results, 
                skip this tool call and continue with subsequent tool call tasks.
                """, chatOptions);

        ChatClient chatClient = builder.build();
        ChatResponse chatResponse = chatClient
                .prompt(prompt)
                .user("What directories are under the 'code' directory, and what is the purpose of each directory?")
                .call()
                .chatResponse();
        System.out.print(chatResponse.getResult().getOutput().getText());

        Scanner scanner = new Scanner(System.in);
        while (chatResponse.hasToolCalls()) {
            for (Generation generation : chatResponse.getResults()) {
                for (AssistantMessage.ToolCall toolCall : generation.getOutput().getToolCalls()) {
                    System.out.printf("%s (tool call %s)\n", generation.getOutput().getText(), toolCall.toString());
                }
            }
            System.out.print("Allow to execute tool calling ?(y/n): ");
            String confirm = scanner.nextLine().trim().toLowerCase();
            if ("y".equals(confirm)) {
                ToolExecutionResult toolExecutionResult = toolCallingManager.executeToolCalls(prompt, chatResponse);
                prompt = new Prompt(toolExecutionResult.conversationHistory(), chatOptions);
                chatResponse = chatClient.prompt(prompt).call().chatResponse();
                System.out.println(chatResponse.getResult().getOutput().getText());
            } else {
                // TODO Reject tool calling
                // Expecting some methed like `toolCallingManager.rejectToolCalls(prompt, chatResponse);`
                System.out.println("User cancel the tool calling operation");
            }
        }
        System.out.println(chatResponse.getResult().getOutput().getText());
        System.out.println("Finished");
    }
}

The output logs of LLM are currently similar to the content below: when tool calling is not executed, the system repeatedly requests the execution of the same tool calling.

 (tool call ToolCall[id=019a7c85fcdec2d390a2ba15d21b7534, type=function, name=list_directory, arguments={"path": "/path1"}])
Allow to execute tool calling ?(y/n): y

 (tool call ToolCall[id=019a7c86356f17e95d977e986f462643, type=function, name=list_directory, arguments={"path": "/path2"}])
Allow to execute tool calling ?(y/n): y

 (tool call ToolCall[id=019a7c866b12e22800aa8de2a659a339, type=function, name=list_directory, arguments={"path": "/path3"}])
Allow to execute tool calling ?(y/n): n
User cancel the tool calling operation

 (tool call ToolCall[id=019a7c866b12e22800aa8de2a659a339, type=function, name=list_directory, arguments={"path": "/path3"}])
Allow to execute tool calling ?(y/n): n
User cancel the tool calling operation

 (tool call ToolCall[id=019a7c866b12e22800aa8de2a659a339, type=function, name=list_directory, arguments={"path": "/path3"}])
Allow to execute tool calling ?(y/n): n
User cancel the tool calling operation

Is there a possibility to add a method similar to toolCallingManager.rejectToolCalls(prompt, chatResponse) for rejecting tool calls?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ToolCallingManager and its executeToolCalls method, then read the User-Controlled Tool Execution documentation linked in the issue. Trace how the conversation history is updated after execution and define the rejection path so a declined call does not repeat indefinitely; done means the flow can continue or finish without re-requesting the rejected tool call.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
ai
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.