spring-projects / spring-projects/spring-ai

Infinite tool calling loop with tool_choice: function

Open
#4,464 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Setting an OpenAi's tool_choice option to "function" triggers an infinite loop of llm trying to call this function.

Looks like when spring-ai calls llm with a tool execution result it copies chatOptions with tool_choice set to function, which forces model to call the same tool again.

I've found this: https://community.openai.com/t/infinite-loop-with-tool-choice-required-or-type-function/755129

Ive tried to reset the tool_choice option in debugger before options are copied for sending a tool call result (at the bottom of org.springframework.ai.openai.OpenAiChatModel#createRequest) - looks like it helped llm to exit loop. But i didnt find an intended way to get around this problem. So maybe im doing something wrong.

Maybe disbling internal toolExecution will help - so i would be able to create OpenAiChatOptions with tool_choice - auto while construncting a message with tool execution result. But it would be nice to continue using internalToolExecution.

I've also encontered similar behaviour with DeepSeek's toolChoice. Didn't try to reset the tool_choice though.

im using spring-ai version 1.0.2

public class ToolChoiceTest {

public void test() {
	try {
		String openaiApiKey = System.getenv("OPENAI_API_KEY");
		var openAiApi = OpenAiApi.builder().apiKey(openaiApiKey).build();
		Object toolChoice = OpenAiApi.ChatCompletionRequest.ToolChoiceBuilder.FUNCTION("getMarineYetiDescription");
		var openAiChatOptions = OpenAiChatOptions.builder().model("gpt-5").toolChoice(toolChoice).build();
		var chatModel = OpenAiChatModel.builder().openAiApi(openAiApi).defaultOptions(openAiChatOptions).build();
		var chatClient = ChatClient.builder(chatModel).defaultTools(this).build();
		var prompt = new Prompt("What is the color of a marine yeti?");
		String response = chatClient.prompt(prompt).call().content();
		System.out.println(response);
	} catch (Exception exception) {
		System.err.println(exception);
	}
}

@Tool(description = "Get description of marine yeti")
public String getMarineYetiDescription() {
	return "Marine yeti is orange";
}

public static void main(String[] args) {
	new ToolChoiceTest().test();
}

}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at org.springframework.ai.openai.OpenAiChatModel#createRequest, where the issue reports that chat options are copied while sending a tool execution result. Reproduce the behavior with the provided ToolChoiceTest and inspect how tool_choice is handled during internal tool execution. Done means a forced function choice does not cause the same tool call to repeat indefinitely.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
ai, api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.