spring-projects / spring-projects/spring-ai

Gemini and function calling: function call may be discarded.

Open
#2,499 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Bug description

This is linked to the PR https://github.com/spring-projects/spring-ai/pull/2029 where I try to fix this.

The gemini can respond multiple part, some containing test, some tool call. A response can be like:

part 1: message: " I understand what you want to do, I will check ..."
part 2: function call
part 3: message "if it is not enough you will need to check yourself .... "

Today, the response is processed by spring-ai in a way that function are executed only if all part of the response are function call.

It seems it is ok to switch the code https://github.com/spring-projects/spring-ai/blob/4fc6edd80c42801ab8aec6530c34a32c73604390/models/spring-ai-vertex-ai-gemini/src/main/java/org/springframework/ai/vertexai/gemini/VertexAiGeminiChatModel.java#L598 to use 'anyMatch' instead of 'allMatch' to check for function call in the response. As long there is a function call in the response, it should be executed.

Also, the response send back to the system is filtered, I think the response should send back all part returned by the API ( https://github.com/spring-projects/spring-ai/blob/4fc6edd80c42801ab8aec6530c34a32c73604390/models/spring-ai-vertex-ai-gemini/src/main/java/org/springframework/ai/vertexai/gemini/VertexAiGeminiChatModel.java#L600 )

Environment

Spring-ai 1.0.0-M6
Gemini api (2.0 flash)

Steps to reproduce

Use springAI with a gemini model, add the tool "CurrentWeatherService" and prompt:

The procedure can be either to check the temperature for a city and then if the temperature you will need to update the fan speed from 0-100 depending on the temperature 0-30. Explain the procedure is more clean word and process it for the city Tokyo

It should trigger a response with 2 parts: 1 small message and 1 function call.

Expected behavior

SpringAI should execute function if there is one in a part of the response.

Also, the return should contains all the generation and all the part. Currently in the event of I push a prompt, genAI ask for a function call, the prompt with the function execution is pushed, only the last generation is send back, which means if we continue the chat, some data are missing.

Minimal Complete Reproducible example

Use springAI with a gemini model, add the tool "CurrentWeatherService" and prompt:

The procedure can be either to check the temperature for a city and then if the temperature you will need to update the fan speed from 0-100 depending on the temperature 0-30. Explain the procedure is more clean word and process it for the city Tokyo

It should trigger a response with 2 parts: 1 small message and 1 function call.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in models/spring-ai-vertex-ai-gemini/src/main/java/org/springframework/ai/vertexai/gemini/VertexAiGeminiChatModel.java around lines 598-600, then reproduce with the CurrentWeatherService tool and the Tokyo prompt. Verify behavior for a response containing both text and a function call, including whether all returned generations and parts remain available after tool execution.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
ai
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.