spring-projects / spring-projects/spring-ai
Supporting multiple LLMs in auto-config
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
Currently, proprietary or open models running in Google Vertex can be accessed through the Vertex Gemini module, respectively using the OpenAI API.
#################################
# Google Vertex AI Gemini
#################################
spring.ai.vertex.ai.gemini.project-id=<project>
spring.ai.vertex.ai.gemini.location=us-central1
spring.ai.vertex.ai.gemini.chat.options.model=gemini-2.0-flash-001
spring.ai.vertex.ai.gemini.transport=grpc
#################################
# OpenAI API VertexAI
#################################
spring.ai.openai.api-key=abc123
spring.ai.openai.vertex.ai.gemini.project-id=<project>
spring.ai.openai.vertex.ai.gemini.location=us-central1
spring.ai.openai.vertex.ai.chat.options.model=meta/llama3-405b-instruct-maas
spring.ai.openai.vertex.ai.chat.base-url=<baseURL>
spring.ai.openai.vertex.ai.chat.completions-path=/chat/completions
spring.ai.openai.vertex.ai.chat.options.max-tokens=1024
The problem - there is no ability to auto-config more than 1 Gemini or open model in Vertex at this time.
The proposal - introduce the ability to add a in the auto-config, to allow for multiple configurations
spring.ai.vertex.ai.gemini.<gemini-flash>.project-id=<project>
spring.ai.vertex.ai.gemini.<gemini-flash>.location=us-central1
spring.ai.vertex.ai.gemini.<gemini-flash>.chat.options.model=gemini-2.0-flash-001
spring.ai.vertex.ai.gemini.<gemini-flash>.transport=grpc
spring.ai.vertex.ai.gemini.<gemini-pro>.project-id=<project>
spring.ai.vertex.ai.gemini.<gemini-pro>.location=us-central1
spring.ai.vertex.ai.gemini.<gemini-pro>.chat.options.model=gemini-1.5-001
spring.ai.vertex.ai.gemini.<gemini-pro>.transport=grpc
Workaround - not using auto-config and manually configuring each model in use
CC @tzolov
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the Vertex Gemini module and the OpenAI API VertexAI auto-configuration described in the issue. Determine how the current single-model properties are bound and instantiated, then verify that separate model-family properties can create multiple configurations without breaking existing configuration behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, java, spring-boot
- Domain
- ai, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100