spring-projects / spring-projects/spring-ai

Support async call by return `Mono<ChatResponse>`

Open
#538 5 comments 7 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

chat client
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Please do a quick search on GitHub issues first, the feature you are about to request might have already been requested.

Expected Behavior

The call method of ChatClient supports returning Mono objects to achieve asynchronous calls.

For example:

public interface AsyncChatClient extends ModelClient<Prompt, ChatResponse> {
    @Override
    Mono<ChatResponse> call(Prompt prompt);
}

Current Behavior

Currently, the call method of AzureOpenAiChatClient does not support asynchronous calls. Only the stream method can return a Flux async object, but Flux represents multiple results. I need Mono object which it represent a single result.

Context

I used com.azure:azure-ai-openai SDK before. This SDK provides two type client: OpenAIClient and OpenAIAsyncClient. OpenAIAsyncClient provides complete Mono async support for a single result. But when I migrated to Spring AI, I found that Mono support was not provided.

image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the ModelClient<Prompt, ChatResponse> call method and AzureOpenAiChatClient, then compare them with the existing stream method that returns Flux. Check how the Spring AI client APIs represent single versus multiple asynchronous results. Done means the call path supports returning Mono while the existing streaming behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.