spring-projects / spring-projects/spring-ai
Unable to retrieve the reasoning content from the OpenAiChatModel during streaming calls
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
Bug description
When chatting with the model, although the thinking switch is turned on, the thinking content still cannot be read from the response.
As shown in the figure above, the additionalProperties.reasoning_content field has a value in the raw response content.
After investigation, I found the following issue:
the OpenAiChatModel#getReasoningContent method reads reasoning-content from ChatCompletionMessage#_additionalProperties, but no value is set for the message in OpenAiChatModel.ChunkMerge#chunkToChatCompletion, so the reasoning content cannot be retrieved.
Environment
- Spring AI: 2.0.0
- Java: 25
Steps to reproduce
- Configure OpenAI and turn on the thinking switch
spring:
ai:
openai:
base-url: **
api-key: **
chat:
model: **
extra-body:
enable_thinking: true
- Controller
package cn.com.deloitte.amsagent.ai.controller;
import io.swagger.v3.oas.annotations.Operation;
import lombok.*;
import lombok.extern.slf4j.Slf4j;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.messages.AssistantMessage;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Flux;
import java.util.Objects;
@RestController
@RequestMapping("/chat")
@RequiredArgsConstructor
@Slf4j
public class ChatController {
private final ChatClient chatClient;
@PostMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Message> stream(@RequestParam String question, @RequestParam(required = false) String conversationId) {
return this.chatClient.prompt()
.user(question)
.stream()
.chatResponse()
.map(response -> {
Message msg = response.getResults().stream().map(result -> {
AssistantMessage output = result.getOutput();
Message.MessageBuilder builder = Message.builder().content(output.getText());
String reasoning = (String) output.getMetadata().get("reasoningContent");
if (reasoning == null || reasoning.isEmpty()) {
// always true
return builder.build();
}
// never
log.info("reasoning: {}", reasoning);
builder.reasoningContent(reasoning);
return builder.build();
}).reduce(Message::merge).orElse(Message.builder().build());
// the reasoningContent is always null
log.info("reasoningContent: {}", msg.getReasoningContent());
if (!Objects.isNull(msg.getReasoningContent())) {
// never
throw new IllegalArgumentException("The reasoningContent has non-null value.");
}
return msg;
});
}
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public static class Message {
private String content;
private String reasoningContent;
public Message merge(Message msg) {
if (content == null) {
content = msg.content;
} else if (msg.getContent() != null) {
content = content + msg.content;
}
if (reasoningContent == null) {
reasoningContent = msg.reasoningContent;
} else if (msg.reasoningContent != null) {
reasoningContent = reasoningContent + msg.reasoningContent;
}
return this;
}
}
}
-
Start the application
-
Call the api
curl -X POST "http://127.0.0.1:8080/chat/stream?question=Why%20there%20isnot%20any%20thinking%20content?"
Any more questions
Expected behavior
I expect an IllegalArgumentException to be thrown when the API is called. However, no matter how I modify thequestion parameter, the exception is never thrown.
Minimal Complete Reproducible example
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with OpenAiChatModel#getReasoningContent and OpenAiChatModel.ChunkMerge#chunkToChatCompletion, then trace how additionalProperties from streaming responses reaches ChatCompletionMessage. Reproduce the issue with enable_thinking enabled and verify that reasoning content is exposed through the response metadata during streaming calls.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100