spring-projects / spring-projects/spring-ai

Response from `ChatClient` Ends Before Entire Response Received (Observed w/Amazon Nova Pro)

Open
#3,871 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Bug description
For some requests to the model the response from the ChatClient ends before the entire response from the model is received. I've noticed this with Nova Pro (2 out of five times, maybe) but this may also effect other models.

My prompt asks the model to evaluate a set of data against a series of rules. When this issue occurs, I will receive only the first response from the model and then the ChatClient decides that's all of the responses. I can tell it's only the first response because it includes the <thinking>...</thinking> block.

I tested with both the call and the stream methods (collecting all of the response from the stream) and have noticed the same behavior with both.

2025-07-21T09:26:44.645-04:00 DEBUG 22025 --- [notecomp] [        async-3] [c16cc2e687ed81d07869770eacba8df0-b87a3db9b2ae1ca0] o.s.a.b.converse.BedrockProxyChatModel   : ConverseResponse: ConverseResponse(Output=ConverseOutput(Message=Message(Role=assistant, Content=[ContentBlock(Text=<thinking> 
To evaluate the data against the provided rule set, I need to identify which rules are satisfied by the information in the data. I will go through each rule and check if there is any relevant information in the data that satisfies the rule. If a rule is satisfied, I will provide up to three citations that support the rule. If a rule is not satisfied, I will only record the rule's ID and indicate that it was not satisfied.

Here are the steps I will follow:
1. Read through the data and extract relevant information.
2. Match the extracted information against each rule in the rule set.
3. For each rule that is satisfied, create citations with the required details.
4. For each rule that is not satisfied, record the rule's ID and indicate that it was not satisfied.
5. Format the evaluation according to the provided JSON schema.
</thinking>

)])), StopReason=end_turn, Usage=TokenUsage(InputTokens=15369, OutputTokens=192, TotalTokens=15561), Metrics=ConverseMetrics(LatencyMs=5650))

Environment
I'm using OpenJDK 21.0.7 with Spring AI 1.1.0-SNAPSHOT. I'm using the Bedrock Converse API.

Steps to reproduce
Make a call to Bedrock with Nova Pro that will likely take multiple turns (i.e. includes tool use).

Expected behavior
I expected to get the complete model response every time. Maybe two out of every five times I only received the first response from the model.

Minimal Complete Reproducible example
If this would be helpful, please let me know. I would also be willing to submit a PR if someone has a hint as to where in the code things might be going wrong. 🙏

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the BedrockProxyChatModel entry point shown in the log and reproduce with the Bedrock Converse API, Nova Pro, and a tool-using request. Compare call with stream collection and trace how the response is terminated after the thinking block. Done means the complete model response is returned consistently, including content after that block.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java
Domain
ai, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.