spring-projects / spring-projects/spring-ai

Title: GoogleGenAiChatModel: enabling thinkingBudget/includeThoughts causes tool calls to silently not fire (model narrates intent in text instead of emitting toolCalls)

Open
#6,579 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Title: GoogleGenAiChatModel: enabling thinkingBudget/includeThoughts causes tool calls to silently not fire (model narrates intent in text instead of emitting toolCalls)
Environment:

Spring AI version: 1.1.6
Module: spring-ai-starter-model-google-genai
Model: gemini-2.5-flash
Java 21, Spring Boot 4.1.0

Description:
When GoogleGenAiChatOptions is configured with thinkingBudget(1024) and includeThoughts(true) alongside .tools(...) on a ChatClient request, the model correctly reasons about which tool to call — visibly narrating its decision in the response text field — but never emits the corresponding structured toolCalls entry. As a result, DefaultToolCallingManager never detects or executes the tool, and the tool call is silently dropped. finishReason is STOP (not truncated), so this isn't a token-limit/truncation issue — the model appears to substitute the narration for the actual function-call action.
Disabling thinking (thinkingBudget(0), includeThoughts(false)) on the identical request, with the identical tools and prompt, resolves the issue immediately — the model correctly emits a populated toolCalls array and the tool executes as expected.
Steps to reproduce:

Configure a ChatClient with Google GenAI, gemini-2.5-flash, and register two simple @Tool-annotated methods (e.g. getStock(String productName), getPrice(String productName)).
Set request options:

java GoogleGenAiChatOptions.builder()
.thinkingBudget(1024)
.includeThoughts(true)
.build()

Send a prompt clearly requiring a tool call, e.g. "stock for car".
Attach SimpleLoggerAdvisor and inspect the response.

Expected behavior:
toolCalls in the response contains a populated entry for getStock with productName: "car", and the tool executes.
Actual behavior:
toolCalls: []. The text field instead contains the model's internal reasoning, e.g.:
"Analyzing the User's Request

Okay, so the user wants the stock level for "car." ... I've got the getStock tool
available... Therefore, the most logical action is to use the getStock tool and
feed it "car" as the productName... Simple as that."
finishReason: STOP, completionTokens: 14, thoughtsTokenCount: 97 — confirming the model completed normally rather than being cut off.
Workaround:
Setting thinkingBudget(0) and includeThoughts(false) on the same request resolves the issue completely — tool calls fire correctly.
Suggested area to investigate:
Possibly in how GoogleGenAiChatModel/GoogleGenAiChatResponseMetadata parses the raw Gemini API response when both thinkingConfig and tools are present in the request — the model's function-call Part may not be getting correctly extracted/mapped into AssistantMessage.toolCalls when a thought Part precedes it in the response, or the request itself may not be correctly signaling function-calling mode is expected once thinking is enabled.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with GoogleGenAiChatModel and GoogleGenAiChatResponseMetadata, focusing on how responses containing thought parts and tools are parsed. Reproduce the issue with gemini-2.5-flash, thinkingBudget(1024), includeThoughts(true), and the two sample tools, then compare it with thinking disabled. Done means the response contains the getStock tool call and the tool executes instead of the reasoning appearing only in text.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring, spring-boot
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.