spring-projects / spring-projects/spring-ai
Can I contribute vLLM model support to Spring AI?
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
Expected Behavior
I would like to contribute a new model integration for vLLM, a fast and memory-efficient inference engine for LLMs.
This new model implementation would enable Spring AI users to connect to a vLLM backend more directly and cleanly, similar to existing integrations with OpenAI and Hugging Face.
Current Behavior
Currently, Spring AI does not offer native support for vLLM.
While it is technically possible to connect to a vLLM backend by customizing the OpenAI model implementation, this approach is not ideal—it requires workarounds and results in less maintainable and reusable code.
Context
I'm currently working on a project that uses vLLM for high-performance inference and would benefit from tighter integration with Spring AI’s abstractions and tooling.
Before proceeding with the implementation and contributing a PR, I’d like to check if the Spring AI team is open to a contribution that adds dedicated support for vLLM.
Are there any existing plans or previous discussions around this? If not, I’d be happy to explore it further and share a prototype.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing Spring AI's existing OpenAI and Hugging Face integrations, then study the vLLM backend API and its compatibility with those abstractions. The issue does not name specific files or tests; done would require agreement on the integration scope and a maintainable native vLLM model implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spring
- Domain
- ai, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100