spring-projects / spring-projects/spring-ai
`ConverseChatResponseStream` drops `reasoningContent`, breaking extended thinking with tool use when streaming
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
Bug description
#6414 fixed extended thinking with tool use on Bedrock Converse for call(), but the streaming path still has the original bug from #6413. ConverseChatResponseStream was not in scope of that fix, so stream() with thinking and tools still fails with a ValidationException on the tool-result turn.
Two things have to change, and either one alone leaves the feature broken.
1. Reasoning deltas are dropped on arrival. ConverseChatResponseStream.visitContentBlockDelta (lines 112-122) matches only tool use and text:
if (toolCallBuilder != null) {
toolCallBuilder.delta(event.delta().toolUse().input());
}
else if (ContentBlockDelta.Type.TEXT.equals(event.delta().type())) {
this.emitChatResponse(new Generation(AssistantMessage.builder().content(event.delta().text()).build()));
}
A reasoning delta arrives on its own contentBlockIndex, never present in toolUseMap, with type REASONING_CONTENT. Both branches miss and it falls out of the method silently. The data is on the wire: bedrockruntime 2.51.2 delivers ReasoningContentBlockDelta with text(), signature() and redactedContent().
2. MessageAggregator discards the subclass. Lines 176-189 rebuild the assistant message with AssistantMessage.builder(), keeping text, properties and tool calls only. So a BedrockAssistantMessage emitted into the stream still reaches the tool loop as a plain AssistantMessage, and the instanceof BedrockAssistantMessage replay guard #6414 added to createRequest can never match. This is the same class of problem #6703 fixed for Prompt.instructionsCopy(), and it sits on the path every streaming tool round takes through ToolCallingAdvisor.
Application code cannot work around either one: BedrockAssistantMessage and BedrockReasoningContent are package-private, and ConverseChatResponseStream is constructed with new inside internalStream with no injection point.
Environment
- Spring AI: 2.0.1 (
ConverseChatResponseStream.javais byte-identical onmain) - Model:
us.anthropic.claude-sonnet-4-6(any thinking-capable Claude model on Bedrock) - Java: 25
- AWS SDK
bedrockruntime2.51.2 viaspring-ai-starter-model-bedrock-converse - No vector store
Steps to reproduce
- Build a
BedrockProxyChatModelfor a thinking-capable Claude model. - Enable extended thinking via
BedrockChatOptions.requestParameters(reaches ConverseadditionalModelRequestFields), withtemperature1.0. - Register a tool and send a prompt that triggers a tool call.
- Stream with
chatModel.stream(prompt), aggregate, execute the tool calls, stream again with the resulting history.
The first response carries reasoningContent and toolUse. The second request throws ValidationException, because the rebuilt assistant turn has toolUse with no preceding signed reasoningContent. The same flow through call() passes on 2.0.1.
Expected behavior
- Reasoning deltas are accumulated per content-block index (text, signature, and
redactedContentwhere present) and preserved on the assistant message for the completed turn, astoChatResponsealready does forcall(). - That state survives aggregation, so the existing replay in
createRequestre-emits the signed block ahead oftoolUseon the next request.
Minimal Complete Reproducible example
Drop-in @Test for BedrockProxyChatModelIT. This is the existing streamFunctionCallTest (line 328) with a thinking-capable model and thinking parameters added, reusing the same MockWeatherService and @RequiresAwsCredentials setup. It passes with thinking off and fails with thinking on.
@Test
void streamFunctionCallWithThinkingEnabled() {
ToolCallingManager toolCallingManager = DefaultToolCallingManager.builder().build();
UserMessage userMessage = new UserMessage(
"What's the weather like in San Francisco, Tokyo and Paris? Return the result in Celsius.");
List<Message> messages = new ArrayList<>(List.of(userMessage));
var promptOptions = BedrockChatOptions.builder()
.model("us.anthropic.claude-sonnet-4-6")
// Anthropic constraint: temperature must be 1.0 when thinking is enabled
.temperature(1.0)
.requestParameters(Map.of("thinking", Map.of("type", "adaptive", "display", "summarized")))
.toolCallbacks(List.of(FunctionToolCallback.builder("getCurrentWeather", new MockWeatherService())
.description("Get the weather in location. Return temperature in 36°F or 36°C format. Use multi-turn if needed.")
.inputType(MockWeatherService.Request.class)
.build()))
.build();
var prompt = new Prompt(messages, promptOptions);
AtomicReference<ChatResponse> aggregatedRef = new AtomicReference<>();
new MessageAggregator().aggregate(this.chatModel.stream(prompt), aggregatedRef::set).collectList().block();
// Fails on the second iteration with ValidationException: the assistant turn rebuilt from
// the aggregated response carries toolUse with no preceding signed reasoningContent block.
while (aggregatedRef.get().hasToolCalls()) {
ToolExecutionResult toolExecutionResult = toolCallingManager.executeToolCalls(prompt, aggregatedRef.get());
prompt = new Prompt(toolExecutionResult.conversationHistory(), promptOptions);
aggregatedRef.set(null);
new MessageAggregator().aggregate(this.chatModel.stream(prompt), aggregatedRef::set).collectList().block();
}
String content = aggregatedRef.get().getResult().getOutput().getText();
assertThat(content).contains("30", "10", "15");
}
There is no test over ConverseChatResponseStream today, and BedrockConverseUsageAggregationTests.streamWithToolCallUse() is an empty // TODO: Implement the test stub, which is probably why this went unnoticed.
Related: #6413, #6414, #6703, #3388. I could not find an existing report for the streaming path.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ConverseChatResponseStream.visitContentBlockDelta and its aggregation path through MessageAggregator, then inspect the existing call() handling and createRequest replay behavior. Run the supplied streamFunctionCallWithThinkingEnabled reproduction in BedrockProxyChatModelIT, or add focused coverage where appropriate; done means reasoning text, signatures, and redacted content survive streaming and aggregation so the subsequent tool-result request succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100