spring-projects / spring-projects/spring-ai

ChunkMerger.chunkToChatCompletion throws OpenAIInvalidDataException when a streamed chunk omits `id` or `model` (only `created` is guarded)

Open Beginner friendly
#6,928 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Bug description

OpenAiChatModel.ChunkMerger.chunkToChatCompletion reads three required fields off
ChatCompletionChunk, but only guards one of them:

// OpenAiChatModel.java:1326-1333 (2.0.1)                                                                                                                                                                                                                       
return ChatCompletion.builder()                                                                                                                                                                                                                                 
    .id(chunk.id())               // unguarded                                                                                                                                                                                                                  
    .choices(choices)                                                                                                                                                                                                                                           
    .created(getCreated(chunk))   // guarded                                                                                                                                                                                                                    
    .model(chunk.model())         // unguarded                                                                                                                                                                                                                  
    .usage(...)                                                                                                                                                                                                                                                 
    .putAllAdditionalProperties(chunk._additionalProperties())                                                                                                                                                                                                  
    .build();                                                                                                                                                                                                                                                   
// OpenAiChatModel.java:1340-1347                                                                                                                                                                                                                               
private static long getCreated(ChatCompletionChunk chunk) {                                                                                                                                                                                                     
    try {                                                                                                                                                                                                                                                       
        return chunk.created();                                                                                                                                                                                                                                 
    }                                                                                                                                                                                                                                                           
    catch (OpenAIInvalidDataException ex) {                                                                                                                                                                                                                     
        return 0L;                                                                                                                                                                                                                                              
    }                                                                                                                                                                                                                                                           
}                                                                                                                                                                                                                                                               

All three accessors go through JsonField.getRequired(...) in openai-java-core and therefore
throw OpenAIInvalidDataException when the field is absent (verified against
openai-java-core 4.49.0 bytecode: ChatCompletionChunk.id(), .model() and .created() all
call JsonField.getRequired$openai_java_core).

The existence of getCreated() shows the code already anticipates providers that omit required
fields — the same defensive treatment is simply missing for id and model.

Impact: with an OpenAI-compatible gateway whose final (usage-only) chunk carries no id, the
entire stream fails after the assistant message has been fully delivered. In our case 95
content deltas had already been emitted and the answer was complete; the exception surfaces at the
very end, MessageAggregator logs Aggregation Error, and the Flux terminates with onError.
Downstream that turns a fully successful request into a failed one (error event pushed to clients,
error-rate metrics polluted).

Environment
  • Spring AI 2.0.1
  • openai-java-core 4.49.0
  • Spring Boot 4.1.0, Java 25
  • Model: a DeepSeek model served through a third-party OpenAI-compatible gateway
    (ChatModel = OpenAiChatModel, streaming, tool calling enabled)
Steps to reproduce

Any OpenAI-compatible endpoint whose final SSE chunk omits id triggers it. A unit test against
ChunkMerger is probably the cleanest repro: feed it a ChatCompletionChunk whose _id() is
absent (e.g. deserialized from {"choices":[],"usage":{...}}) and observe the exception.

Stack trace
com.openai.errors.OpenAIInvalidDataException: `id` is not set                                                                                                                                                                                                   
    at com.openai.core.JsonField.getRequired$openai_java_core(Values.kt:174)                                                                                                                                                                                    
    at com.openai.models.chat.completions.ChatCompletionChunk.id(ChatCompletionChunk.kt:91)                                                                                                                                                                     
    at org.springframework.ai.openai.OpenAiChatModel$ChunkMerger.chunkToChatCompletion(OpenAiChatModel.java:1327)                                                                                                                                               
    at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onNext(FluxMap.java:208)                                                                                                                                                                         
    at reactor.core.publisher.FluxBufferPredicate$BufferPredicateSubscriber.onNextNewBuffer(FluxBufferPredicate.java:316)                                                                                                                                       
    ...                                                                                                                                                                                                                                                         
    at com.openai.core.http.TrackedHandler.onNext(PhantomReachableClosingAsyncStreamResponse.kt:53)                                                                                                                                                             
Suggested fix

Apply the getCreated pattern to the other two required fields, e.g.

private static String getId(ChatCompletionChunk chunk) {                                                                                                                                                                                                        
    try {                                                                                                                                                                                                                                                       
        return chunk.id();                                                                                                                                                                                                                                      
    }                                                                                                                                                                                                                                                           
    catch (OpenAIInvalidDataException ex) {                                                                                                                                                                                                                     
        return "";                                                                                                                                                                                                                                              
    }                                                                                                                                                                                                                                                           
}                                                                                                                                                                                                                                                               

ChatCompletionChunk also exposes _id() / _model() (raw JsonField) and isValid(), so the
check can be done without relying on exceptions if that is preferred. Carrying over the id from an
earlier chunk in the same buffered group would preserve more information than a placeholder, since
ChunkMerger already has the whole group in hand.

Related

Same class, third variant of this failure mode: #6591 (continuation frames with "id": "" treated
as new tool calls, fixed in 2.0.1) was about tool-call merging; this one is about required
top-level fields on the chunk itself. The underlying assumption — that every chunk conforms to the
official OpenAI schema — does not hold for third-party OpenAI-compatible gateways, which is worth
hardening across ChunkMerger rather than field by field.

Workaround

Not interceptable from application code (ChunkMerger sits inside internalStream). We tolerate
it downstream instead: once a chunk has reported a finish_reason other than TOOL_CALLS, the
model has stopped generating and only a usage frame remains, so any later parse failure is treated
as a benign end-of-stream condition (logged at WARN) rather than a request failure.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in OpenAiChatModel.java at ChunkMerger.chunkToChatCompletion and read getCreated alongside the ChatCompletionChunk accessors in openai-java-core. Reproduce or add coverage for a streamed final chunk missing id or model; done means the completed assistant stream no longer terminates with OpenAIInvalidDataException while the existing created handling remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring-boot
Domain
backend-api-design, stream-processing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.