spring-projects / spring-projects/spring-ai

Add Filter for Modifying Response Body in OpenAiApi#chatCompletionStream

Open
#2,402 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Expected Behavior

The OpenAiApi#chatCompletionStream method should provide a way to filter and modify the returned body before it is processed. This would allow developers to adapt responses from API providers that claim to be OpenAI-compatible but have partial deviations.

Current Behavior

Currently, the response from chatCompletionStream is directly processed using ModelOptionsUtils.jsonToObject(content, ChatCompletionChunk.class). However, some third-party API providers do not fully adhere to OpenAI's API response format, causing deserialization failures.

Context

This issue affects users integrating with third-party APIs that claim OpenAI compatibility but have subtle differences in response formatting.

A filtering mechanism would allow developers to preprocess the response body and adjust it before it is parsed into ChatCompletionChunk.

There are currently no known workarounds other than modifying the library code itself.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at OpenAiApi#chatCompletionStream and trace how the response reaches ModelOptionsUtils.jsonToObject(content, ChatCompletionChunk.class). Determine the filtering extension point and define done as allowing third-party response bodies to be adjusted before ChatCompletionChunk processing without breaking existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.