spring-projects / spring-projects/spring-ai

Tool call results returned via ChatResponse lack identification metadata

Open
#3,890 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

We’re currently developing a tool-based intelligent feature using Spring AI and have encountered a limitation.

In the current implementation, OllamaChatModel.internalStream() does not expose any way to identify whether a streamed ChatResponse is generated from a tool_call. As a result, the frontend developers have no way of distinguishing whether a particular response is from a tool or a regular model output, making it difficult to render tool-specific UI behavior.

After reviewing the source code, we noticed that tool-related responses are not marked or separated in the streaming flow, and there's no hook or metadata exposed to help with this.

🙏 Feature Request
Would it be possible to:

Add metadata or a flag in ChatResponse indicating whether it is a tool-generated message?

Or expose tool_calls (as in OpenAI's API spec) via ChatResponse when available?

Or allow developers to customize the parsing of the raw response, for more advanced routing?

This feature would greatly improve the ability to integrate tool responses cleanly into frontend workflows.

Thanks for the great work on Spring AI! Looking forward to your thoughts.

Image

https://www.antgroup.com/news-media/press-releases/1714118400000

data:{"result":{"metadata":{"finishReason":"returnDirect","contentFilters":[],"empty":false},"output":{"messageType":"ASSISTANT","metadata":{"messageType":"ASSISTANT"},"toolCalls":[],"media":[],"text":"[{"productAmount":"888.88","productId":"1000","productName":"productName1"},{"productAmount":"666.88","productId":"1002","productName":"productName2"},{"productAmount":"88.88","productId":"1003","productName":"productName3"}]"}},"metadata":{"id":"","model":"qwen3:30bqwen3:30b","rateLimit":{"requestsReset":"PT0S","tokensReset":"PT0S","requestsLimit":0,"tokensLimit":0,"tokensRemaining":0,"requestsRemaining":0},"usage":{"promptTokens":441,"completionTokens":202,"totalTokens":643},"promptMetadata":[],"empty":false},"results":[{"metadata":{"finishReason":"returnDirect","contentFilters":[],"empty":false},"output":{"messageType":"ASSISTANT","metadata":{"messageType":"ASSISTANT"},"toolCalls":[],"media":[],"text":"[{"productAmount":"888.88","productId":"1000","productName":"productName1"},{"productAmount":"666.88","productId":"1002","productName":"productName2"},{"productAmount":"88.88","productId":"1003","productName":"productName3"}]"}}]}

Image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with OllamaChatModel.internalStream() and the ChatResponse representation to trace how tool-related responses move through the streaming flow. Define and implement a single supported way to expose tool-call identification or metadata, then verify that streamed responses distinguish tool output from regular model output.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, ollama
Domain
ai, api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.