spring-projects / spring-projects/spring-ai
Add Filter for Modifying Response Body in OpenAiApi#chatCompletionStream
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
Expected Behavior
The OpenAiApi#chatCompletionStream method should provide a way to filter and modify the returned body before it is processed. This would allow developers to adapt responses from API providers that claim to be OpenAI-compatible but have partial deviations.
Current Behavior
Currently, the response from chatCompletionStream is directly processed using ModelOptionsUtils.jsonToObject(content, ChatCompletionChunk.class). However, some third-party API providers do not fully adhere to OpenAI's API response format, causing deserialization failures.
Context
This issue affects users integrating with third-party APIs that claim OpenAI compatibility but have subtle differences in response formatting.
A filtering mechanism would allow developers to preprocess the response body and adjust it before it is parsed into ChatCompletionChunk.
There are currently no known workarounds other than modifying the library code itself.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at OpenAiApi#chatCompletionStream and trace how the response reaches ModelOptionsUtils.jsonToObject(content, ChatCompletionChunk.class). Determine the filtering extension point and define done as allowing third-party response bodies to be adjusted before ChatCompletionChunk processing without breaking existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100