spring-projects / spring-projects/spring-ai

Sync call to getEmbeddings via Vector store Document Writer in Reactive workflow

Open
#5,137 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 10h
Merged PRs (30d)
5

Description

Bug description

When using spring webflux it is expected all the downstream calls are async. When using Azure Open AI Embeddings and Vector store to write documents, the default implementation uses getEmbeddingsWithResponse which uses getEmbeddingsSync. This leads java.lang.IllegalStateException: block()/blockFirst()/blockLast() are blocking, which is not supported in thread parallel-7

Stack trace

com.azure.ai.openai.implementation.OpenAIClientImpl.getEmbeddingsWithResponse(OpenAIClientImpl.java:2361)
\tat com.azure.ai.openai.OpenAIClient.getEmbeddingsWithResponse(OpenAIClient.java:168)
\tat com.azure.ai.openai.OpenAIClient.getEmbeddings(OpenAIClient.java:620)
\tat org.springframework.ai.azure.openai.AzureOpenAiEmbeddingModel.lambda$call$1(AzureOpenAiEmbeddingModel.java:143)
\tat io.micrometer.observation.Observation.observe(Observation.java:564)
\tat org.springframework.ai.azure.openai.AzureOpenAiEmbeddingModel.call(AzureOpenAiEmbeddingModel.java:142)
\tat org.springframework.ai.embedding.EmbeddingModel.embed(EmbeddingModel.java:91)
\tat org.springframework.ai.vectorstore.qdrant.QdrantVectorStore.doAdd(QdrantVectorStore.java:179)
\tat org.springframework.ai.vectorstore.observation.AbstractObservationVectorStore.lambda$add$1(AbstractObservationVectorStore.java:85)
\tat io.micrometer.observation.Observation.observe(Observation.java:498)
\tat org.springframework.ai.vectorstore.observation.AbstractObservationVectorStore.add(AbstractObservationVectorStore.java:85)
\tat org.springframework.ai.vectorstore.VectorStore.accept(VectorStore.java:55)
\tat org.springframework.ai.vectorstore.VectorStore.accept(VectorStore.java:40)
\tat org.springframework.ai.document.DocumentWriter.write(DocumentWriter.java:30)

Environment
Spring AI version: 1.1.2,
Java version - 17

Steps to reproduce
Use spring webflux, Azure Open AI Embedding, Qdrant Vector Store.
Write any document using vectorStore.write() method.

Expected behavior
Embedding is fetched using the provided OpenAIAsyncClient rather than OpenAIClient.

Minimal Complete Reproducible example

spring:
  main:
    web-application-type: reactive
  ai:
    vectorstore:
      qdrant:
        host: localhost
        port: 6334
        collection-name: "collection"
        use-tls: false
        initialize-schema: true
    azure:
      openai:
        endpoint: ${azure-openai-endpoint}
        api-key: ${azure-openai-api-key}
        embeddings:
          options:
            deployment-name: "text-embedding-ada-002"
            api-version: "2023-05-15"
vectorStore.write(Document.builder()
                    .text(userMessage.getText())
                    .metadata(metadata)
                    .build();) 

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with AzureOpenAiEmbeddingModel.call and trace the calls through QdrantVectorStore.doAdd, AbstractObservationVectorStore, and DocumentWriter.write using the provided WebFlux reproduction. Confirm where OpenAIClient is selected instead of OpenAIAsyncClient; done means document writing fetches embeddings asynchronously without the blocking exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, java
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.