spring-projects / spring-projects/spring-ai

Streaming `reasoningContent` is exposed as accumulated full text — no incremental API for SSE typewriter UX

Open
#6,833 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Background

Spring AI 2.0.1, spring-ai-starter-model-openai, streaming via ChatClient.stream().chatResponse() against an OpenAI-compatible gateway backed by DeepSeek.

Since #6373 (aligned streaming reasoning_content with non-streaming responses), every intermediate streamed chunk carries the full accumulated reasoning text in the chunk metadata (metadata["reasoningContent"]):

String accumulatedReasoning = reasoningMap.merge(id + ":" + choice.index(),
        getReasoningContent(choice), String::concat);
// ...
REASONING_CONTENT, accumulatedReasoning

I understand the rationale (the final aggregated AssistantMessage must keep the complete reasoning for advisor/chat-memory replay, since providers like DeepSeek require reasoning_content on replayed assistant tool-call messages).

Problem

Streaming the reasoning to a frontend via SSE with a typewriter effect is a first-class need (ChatGPT-style UX). But with the current API there is no incremental reasoning surface:

  • Flux<ChatResponse> chunks expose only the accumulated full text under metadata["reasoningContent"] — the wire delta.reasoning_content is incremental, but it never reaches consumers.
  • Consumers are forced to reconstruct deltas client-side by prefix-stripping the accumulated text (accumulate → splice → trim), which is fragile, duplicated across every app, and easy to get wrong (multi-round tool loops, stream restarts).

This mirrors #5910 (tool-call argument deltas are also swallowed by provider/framework aggregation — milestone 2.1.x).

Expected

Please provide an official incremental reasoning API, e.g. any of:

  1. A per-chunk reasoningDelta field (alongside the accumulated reasoningContent) on the streamed AssistantMessage — mirrors what Vercel AI SDK does with reasoning-delta parts and what OpenAI/DeepSeek already send on the wire.
  2. Or a unified stream-event API covering both reasoning deltas and tool-call arg deltas (reuse the #5910 effort).
  3. At minimum, document the behavior: "intermediate chunks expose accumulated reasoning; derive deltas yourself" and show a canonical example.

Until then, every Spring AI user building a reasoning-aware chat UI has to reinvent the prefix-strip workaround.

Repro context

// 2.0.1, OpenAiChatModel stream()
chatModel.stream(prompt)
    .doOnNext(res -> {
        String full = res.getResult().getOutput().getMetadata()
                         .get("reasoningContent").toString();
        // full grows: "The" -> "The user asks" -> "The user asks what day" ...
    });

Related: #5898, #6230, #6373, #6375, #5910.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with OpenAiChatModel stream() and ChatClient.stream().chatResponse(), then trace how reasoningContent is placed in streamed ChatResponse metadata. Compare the related #5910 and #6373 work before choosing an incremental API shape. Done means streamed consumers can access reasoning deltas while the accumulated reasoning remains available for replay and aggregation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.