spring-projects / spring-projects/spring-ai
IndexOutOfBoundsException in OpenAiChatModel.internalStream when choice.index() != list position
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 5
Description
Bug description
OpenAiChatModel.internalStream() throws IndexOutOfBoundsException when a streaming chunk contains a choice whose index field value does not match its position in the choices array.
At OpenAiChatModel.java:322, the code uses the API-level choice.index() field to index into the raw chunk.choices() list:
"chunkChoice", chunk.choices().get((int) choice.index())
The choice.index() value is the semantic index from the OpenAI API response (e.g., 0 or 1), not the positional index within the choices array. When a chunk contains a single choice with index: 1, chunk.choices() has length 1, but chunk.choices().get(1) is called — causing the IndexOutOfBoundsException.
Environment
- Spring AI version: 2.0.0-M6
- Java version: 25
- LLM provider: OpenAI-compatible API gateway
Steps to reproduce
- Configure
OpenAiChatModelwithstream_options.include_usage: true(the default for streaming) - Use an OpenAI-compatible API gateway that proxies non-OpenAI models
- Send a streaming chat completion request
- The gateway sends chunks where a choice's
indexfield may not match its position in thechoicesarray
Expected behavior
Streaming should complete successfully regardless of the index field values in choices.
Actual Behavior
java.lang.IndexOutOfBoundsException: Index 1 out of bounds for length 1
at java.base/java.util.ArrayList.get(ArrayList.java:428)
at org.springframework.ai.openai.OpenAiChatModel.lambda$internalStream$6(OpenAiChatModel.java:322)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:214)
at java.base/java.util.Collections$2.tryAdvance(Collections.java:5182)
at java.base/java.util.Collections$2.forEachRemaining(Collections.java:5190)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
...
Root Cause Analysis
In OpenAiChatModel.internalStream(), when building generation metadata:
List<Generation> generations = chatCompletion.choices().stream().map(choice -> {
// ...
Map<String, Object> metadata = Map.of(
// ...
"chunkChoice", chunk.choices().get((int) choice.index()) // BUG: uses API index as list position
);
return buildGeneration(choice, metadata, request);
}).toList();
chunkToChatCompletion(chunk) converts chunk choices to ChatCompletion.Choice objects, preserving the original index field from the API response. The code then iterates over chatCompletion.choices() and uses choice.index() to look up the original chunk choice from chunk.choices().
This breaks when choice.index() (API field) ≠ list position, which can happen when:
- The gateway sends chunks with non-zero-based or non-contiguous choice indices
- A usage-only final chunk has fewer choices than expected
- The gateway sends choices in non-sequential order
Suggested Fix
Use the stream iteration index (list position) instead of the API choice.index() field to look up the corresponding chunk choice:
List<ChatCompletion.Choice> choiceList = chatCompletion.choices();
List<Generation> generations = IntStream.range(0, choiceList.size()).mapToObj(i -> {
ChatCompletion.Choice choice = choiceList.get(i);
// ...
Map<String, Object> metadata = Map.of(
// ...
"chunkChoice", chunk.choices().get(i) // use list position, not API index
);
return buildGeneration(choice, metadata, request);
}).toList();
Or alternatively, build a lookup map from index → chunk choice:
Map<Long, ChatCompletionChunk.Choice> chunkChoicesByIndex = chunk.choices().stream()
.collect(Collectors.toMap(ChatCompletionChunk.Choice::index, Function.identity()));
// Then in the mapping:
"chunkChoice", chunkChoicesByIndex.get(choice.index())
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in models/spring-ai-openai/src/main/java/org/springframework/ai/openai/OpenAiChatModel.java at internalStream(), around line 322, and trace how chunkToChatCompletion preserves choice.index(). Reproduce a streaming chunk whose choice index differs from its array position, then verify streaming completes without an IndexOutOfBoundsException and that chunkChoice metadata still maps correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100