spring-projects / spring-projects/spring-ai

`ConverseChatResponseStream` drops `reasoningContent`, breaking extended thinking with tool use when streaming

Open
#6,845 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Bug description

#6414 fixed extended thinking with tool use on Bedrock Converse for call(), but the streaming path still has the original bug from #6413. ConverseChatResponseStream was not in scope of that fix, so stream() with thinking and tools still fails with a ValidationException on the tool-result turn.

Two things have to change, and either one alone leaves the feature broken.

1. Reasoning deltas are dropped on arrival. ConverseChatResponseStream.visitContentBlockDelta (lines 112-122) matches only tool use and text:

if (toolCallBuilder != null) {
	toolCallBuilder.delta(event.delta().toolUse().input());
}
else if (ContentBlockDelta.Type.TEXT.equals(event.delta().type())) {
	this.emitChatResponse(new Generation(AssistantMessage.builder().content(event.delta().text()).build()));
}

A reasoning delta arrives on its own contentBlockIndex, never present in toolUseMap, with type REASONING_CONTENT. Both branches miss and it falls out of the method silently. The data is on the wire: bedrockruntime 2.51.2 delivers ReasoningContentBlockDelta with text(), signature() and redactedContent().

2. MessageAggregator discards the subclass. Lines 176-189 rebuild the assistant message with AssistantMessage.builder(), keeping text, properties and tool calls only. So a BedrockAssistantMessage emitted into the stream still reaches the tool loop as a plain AssistantMessage, and the instanceof BedrockAssistantMessage replay guard #6414 added to createRequest can never match. This is the same class of problem #6703 fixed for Prompt.instructionsCopy(), and it sits on the path every streaming tool round takes through ToolCallingAdvisor.

Application code cannot work around either one: BedrockAssistantMessage and BedrockReasoningContent are package-private, and ConverseChatResponseStream is constructed with new inside internalStream with no injection point.

Environment

  • Spring AI: 2.0.1 (ConverseChatResponseStream.java is byte-identical on main)
  • Model: us.anthropic.claude-sonnet-4-6 (any thinking-capable Claude model on Bedrock)
  • Java: 25
  • AWS SDK bedrockruntime 2.51.2 via spring-ai-starter-model-bedrock-converse
  • No vector store

Steps to reproduce

  1. Build a BedrockProxyChatModel for a thinking-capable Claude model.
  2. Enable extended thinking via BedrockChatOptions.requestParameters (reaches Converse additionalModelRequestFields), with temperature 1.0.
  3. Register a tool and send a prompt that triggers a tool call.
  4. Stream with chatModel.stream(prompt), aggregate, execute the tool calls, stream again with the resulting history.

The first response carries reasoningContent and toolUse. The second request throws ValidationException, because the rebuilt assistant turn has toolUse with no preceding signed reasoningContent. The same flow through call() passes on 2.0.1.

Expected behavior

  • Reasoning deltas are accumulated per content-block index (text, signature, and redactedContent where present) and preserved on the assistant message for the completed turn, as toChatResponse already does for call().
  • That state survives aggregation, so the existing replay in createRequest re-emits the signed block ahead of toolUse on the next request.

Minimal Complete Reproducible example

Drop-in @Test for BedrockProxyChatModelIT. This is the existing streamFunctionCallTest (line 328) with a thinking-capable model and thinking parameters added, reusing the same MockWeatherService and @RequiresAwsCredentials setup. It passes with thinking off and fails with thinking on.

@Test
void streamFunctionCallWithThinkingEnabled() {

	ToolCallingManager toolCallingManager = DefaultToolCallingManager.builder().build();

	UserMessage userMessage = new UserMessage(
			"What's the weather like in San Francisco, Tokyo and Paris? Return the result in Celsius.");

	List<Message> messages = new ArrayList<>(List.of(userMessage));

	var promptOptions = BedrockChatOptions.builder()
		.model("us.anthropic.claude-sonnet-4-6")
		// Anthropic constraint: temperature must be 1.0 when thinking is enabled
		.temperature(1.0)
		.requestParameters(Map.of("thinking", Map.of("type", "adaptive", "display", "summarized")))
		.toolCallbacks(List.of(FunctionToolCallback.builder("getCurrentWeather", new MockWeatherService())
			.description("Get the weather in location. Return temperature in 36°F or 36°C format. Use multi-turn if needed.")
			.inputType(MockWeatherService.Request.class)
			.build()))
		.build();

	var prompt = new Prompt(messages, promptOptions);

	AtomicReference<ChatResponse> aggregatedRef = new AtomicReference<>();
	new MessageAggregator().aggregate(this.chatModel.stream(prompt), aggregatedRef::set).collectList().block();

	// Fails on the second iteration with ValidationException: the assistant turn rebuilt from
	// the aggregated response carries toolUse with no preceding signed reasoningContent block.
	while (aggregatedRef.get().hasToolCalls()) {
		ToolExecutionResult toolExecutionResult = toolCallingManager.executeToolCalls(prompt, aggregatedRef.get());
		prompt = new Prompt(toolExecutionResult.conversationHistory(), promptOptions);
		aggregatedRef.set(null);
		new MessageAggregator().aggregate(this.chatModel.stream(prompt), aggregatedRef::set).collectList().block();
	}

	String content = aggregatedRef.get().getResult().getOutput().getText();
	assertThat(content).contains("30", "10", "15");
}

There is no test over ConverseChatResponseStream today, and BedrockConverseUsageAggregationTests.streamWithToolCallUse() is an empty // TODO: Implement the test stub, which is probably why this went unnoticed.

Related: #6413, #6414, #6703, #3388. I could not find an existing report for the streaming path.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ConverseChatResponseStream.visitContentBlockDelta and its aggregation path through MessageAggregator, then inspect the existing call() handling and createRequest replay behavior. Run the supplied streamFunctionCallWithThinkingEnabled reproduction in BedrockProxyChatModelIT, or add focused coverage where appropriate; done means reasoning text, signatures, and redacted content survive streaming and aggregation so the subsequent tool-result request succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.