spring-projects / spring-projects/spring-ai

Conversion for OpenAI-compliant outputs in Spring AI

Open
#1,379 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement openai
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Expected Behavior

Essentially, title.

Some of the modern UI frameworks for LLMs tend to standardize on using what's called an "OpenAI compliant API" response format, where, essentially, you can use any LLM+server-side framework combination you want as long as you adhere to the OpenAI response format.

Current Behavior

The ChatResponse class is not directly compatible with OpenAI which is, of course, desirable, as Spring AI aims to be a generic framework that abstracts over many different providers with a unified API.

The cost of writing a small "wrapper record" is negligible and it works okay, but, is there some function or method that we can call somewhere in the framework to make the responses be OpenAI-compliant?

Context

Essentially, many clients standardize on OpenAI compliant formats which can make some of the output of SpringAI needing to be adapted. It could make sense to offer this out of the box.

Example of a wrapper record for streaming "OpenAI chunks":

public class OpenAIStreamingResponse {
    public record ChatCompletionChunk(
            String id,
            String object,
            long created,
            String model,
            @JsonProperty("system_fingerprint") String systemFingerprint,
            List<ChunkChoice> choices) {}

    public record ChunkChoice(
            int index, Delta delta, Object logprobs, @JsonProperty("finish_reason") String finishReason) {}

    public record Delta(String role, String content) {}
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ChatResponse class and compare its shape with the OpenAI-compatible streaming wrapper shown in the issue. Determine the supported conversion scope and where it should live; done means Spring AI responses can be consumed in the requested OpenAI-compatible format without each client writing its own wrapper.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.