spring-projects / spring-projects/spring-ai
Infinite tool calling loop with tool_choice: function
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
Setting an OpenAi's tool_choice option to "function" triggers an infinite loop of llm trying to call this function.
Looks like when spring-ai calls llm with a tool execution result it copies chatOptions with tool_choice set to function, which forces model to call the same tool again.
I've found this: https://community.openai.com/t/infinite-loop-with-tool-choice-required-or-type-function/755129
Ive tried to reset the tool_choice option in debugger before options are copied for sending a tool call result (at the bottom of org.springframework.ai.openai.OpenAiChatModel#createRequest) - looks like it helped llm to exit loop. But i didnt find an intended way to get around this problem. So maybe im doing something wrong.
Maybe disbling internal toolExecution will help - so i would be able to create OpenAiChatOptions with tool_choice - auto while construncting a message with tool execution result. But it would be nice to continue using internalToolExecution.
I've also encontered similar behaviour with DeepSeek's toolChoice. Didn't try to reset the tool_choice though.
im using spring-ai version 1.0.2
public class ToolChoiceTest {
public void test() {
try {
String openaiApiKey = System.getenv("OPENAI_API_KEY");
var openAiApi = OpenAiApi.builder().apiKey(openaiApiKey).build();
Object toolChoice = OpenAiApi.ChatCompletionRequest.ToolChoiceBuilder.FUNCTION("getMarineYetiDescription");
var openAiChatOptions = OpenAiChatOptions.builder().model("gpt-5").toolChoice(toolChoice).build();
var chatModel = OpenAiChatModel.builder().openAiApi(openAiApi).defaultOptions(openAiChatOptions).build();
var chatClient = ChatClient.builder(chatModel).defaultTools(this).build();
var prompt = new Prompt("What is the color of a marine yeti?");
String response = chatClient.prompt(prompt).call().content();
System.out.println(response);
} catch (Exception exception) {
System.err.println(exception);
}
}
@Tool(description = "Get description of marine yeti")
public String getMarineYetiDescription() {
return "Marine yeti is orange";
}
public static void main(String[] args) {
new ToolChoiceTest().test();
}
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at org.springframework.ai.openai.OpenAiChatModel#createRequest, where the issue reports that chat options are copied while sending a tool execution result. Reproduce the behavior with the provided ToolChoiceTest and inspect how tool_choice is handled during internal tool execution. Done means a forced function choice does not cause the same tool call to repeat indefinitely.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spring
- Domain
- ai, api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100