spring-projects / spring-projects/spring-ai

Ollama Streaming API does not return 'thinking' metadata in ChatResponse

Open
#4,866 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 10h
Merged PRs (30d)
5

Description

Problem
When using ollamaChatModel.call(), the "thinking" metadata field is correctly returned.
However, when using the streaming API (ollamaChatModel.stream()), the "thinking" field always comes as null.

Code Snippet
Non-streaming version (works):

ChatResponse response = ollamaChatModel.call(...);
String thinking = response.getResult().getMetadata().get("thinking"); // Works

Streaming version (not working):

Flux<ChatResponse> stream = ollamaChatModel.stream(
    new Prompt(
        "Hi",
        OllamaChatOptions.builder()
            .model("gpt-oss:20b")
            .enableThinking()
            .thinkHigh()
            .build()
    )
);

stream.subscribe(response -> {
    String thinking = response.getResult().getMetadata().get("thinking"); //always null
    String content = response.getResult().getOutput().getText();
    System.out.println("[Thinking] " + thinking); //it's null
    System.out.println("[Response] " + content);
});

The thinking field is coming as null.
If I’m missing something or need to enable any additional setting for streaming to return the thinking data, please let me know.

From what I can see, in the non-streaming path
internalCall()
the "thinking" metadata is being extracted and added to the response.

But in the streaming path
internalStream()
that logic seems to be missing, so the thinking field may not be populated.

I’m not sure if this is handled elsewhere — just guessing here.
If its working for others, please let me know.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/OllamaChatModel.java, comparing internalCall() with internalStream() at the locations linked in the issue. Trace how streaming ChatResponse metadata is assembled and verify that streamed responses expose the thinking value when thinking is enabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, ollama
Domain
ai, api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.