Using VertexAI Embeddings model backed by the Predict API over gRPC is extremely slow

Bug description

Hello! We are experimenting with lots of different models etc for both conversation and embedding, and we ran into an issue using the spring-ai-starter-model-vertex-ai-embedding.

We set up the library mostly using auto-configuration, except we use our own GoogleCredentials using the example in the docs. Under the hood, it looks like the library is using the Google SDK predict API over gRPC.

When we call something that uses embeddings, the call takes over 10 seconds. I tried to drill down why that is using the profiler, and this is all I could find:

In the screenshot, you will see that I ran 3 similarity search calls, each that took more than 10 seconds (total of 30k milliseconds in the right column). All of them just seem to be parked for almost all of that time. I doubt the gRPC call (that I assume happens on another thread) should take over 10 seconds.

It also could be that maybe this is completely outside of your control. I just thought I would mention it here.

Environment
Spring AI version: 1.0.1
Java version: 21 (with Kotlin 2.1)

Steps to reproduce

• Use the spring-ai-starter-model-vertex-ai-embedding
• Configure it to use service-account credentials from a JSON file as is stated in the docs: https://docs.spring.io/spring-ai/reference/api/embeddings/vertexai-embeddings-text.html#_load_credentials_from_a_google_service_account
• Run some high level command that triggers an embedding (in our case it looks like this)

val hits = vectorStore.similaritySearch(
    SearchRequest.builder()
        .query(query)
        .topK(10)
        .build()
)

Expected behavior
When I run the predict API myself using their regular HTTP equivalent, making these predictions takes sub-200ms.

Of course - I did this attempt completely bypassing the Google SDK - simply using the Spring RestClient instead.

Minimal Complete Reproducible example
I can return to this later if needed.

Start by reproducing the delay with spring-ai-starter-model-vertex-ai-embedding, service-account credentials, and the similaritySearch call, then profile the gRPC Predict API path and compare it with the RestClient HTTP path. Done means identifying the source of the latency and either correcting it or documenting a confirmed limitation with supporting measurements.