spring-projects / spring-projects/spring-ai
Streaming `reasoningContent` is exposed as accumulated full text — no incremental API for SSE typewriter UX
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
Background
Spring AI 2.0.1, spring-ai-starter-model-openai, streaming via ChatClient.stream().chatResponse() against an OpenAI-compatible gateway backed by DeepSeek.
Since #6373 (aligned streaming reasoning_content with non-streaming responses), every intermediate streamed chunk carries the full accumulated reasoning text in the chunk metadata (metadata["reasoningContent"]):
String accumulatedReasoning = reasoningMap.merge(id + ":" + choice.index(),
getReasoningContent(choice), String::concat);
// ...
REASONING_CONTENT, accumulatedReasoning
I understand the rationale (the final aggregated AssistantMessage must keep the complete reasoning for advisor/chat-memory replay, since providers like DeepSeek require reasoning_content on replayed assistant tool-call messages).
Problem
Streaming the reasoning to a frontend via SSE with a typewriter effect is a first-class need (ChatGPT-style UX). But with the current API there is no incremental reasoning surface:
Flux<ChatResponse>chunks expose only the accumulated full text undermetadata["reasoningContent"]— the wiredelta.reasoning_contentis incremental, but it never reaches consumers.- Consumers are forced to reconstruct deltas client-side by prefix-stripping the accumulated text (accumulate → splice → trim), which is fragile, duplicated across every app, and easy to get wrong (multi-round tool loops, stream restarts).
This mirrors #5910 (tool-call argument deltas are also swallowed by provider/framework aggregation — milestone 2.1.x).
Expected
Please provide an official incremental reasoning API, e.g. any of:
- A per-chunk
reasoningDeltafield (alongside the accumulatedreasoningContent) on the streamedAssistantMessage— mirrors what Vercel AI SDK does withreasoning-deltaparts and what OpenAI/DeepSeek already send on the wire. - Or a unified stream-event API covering both reasoning deltas and tool-call arg deltas (reuse the #5910 effort).
- At minimum, document the behavior: "intermediate chunks expose accumulated reasoning; derive deltas yourself" and show a canonical example.
Until then, every Spring AI user building a reasoning-aware chat UI has to reinvent the prefix-strip workaround.
Repro context
// 2.0.1, OpenAiChatModel stream()
chatModel.stream(prompt)
.doOnNext(res -> {
String full = res.getResult().getOutput().getMetadata()
.get("reasoningContent").toString();
// full grows: "The" -> "The user asks" -> "The user asks what day" ...
});
Related: #5898, #6230, #6373, #6375, #5910.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with OpenAiChatModel stream() and ChatClient.stream().chatResponse(), then trace how reasoningContent is placed in streamed ChatResponse metadata. Compare the related #5910 and #6373 work before choosing an incremental API shape. Done means streamed consumers can access reasoning deltas while the accumulated reasoning remains available for replay and aggregation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spring
- Domain
- ai, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100