spring-projects / spring-projects/spring-ai

Can I contribute vLLM model support to Spring AI?

Open
#3,882 4 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Expected Behavior

I would like to contribute a new model integration for vLLM, a fast and memory-efficient inference engine for LLMs.
This new model implementation would enable Spring AI users to connect to a vLLM backend more directly and cleanly, similar to existing integrations with OpenAI and Hugging Face.

Current Behavior

Currently, Spring AI does not offer native support for vLLM.
While it is technically possible to connect to a vLLM backend by customizing the OpenAI model implementation, this approach is not ideal—it requires workarounds and results in less maintainable and reusable code.

Context

I'm currently working on a project that uses vLLM for high-performance inference and would benefit from tighter integration with Spring AI’s abstractions and tooling.
Before proceeding with the implementation and contributing a PR, I’d like to check if the Spring AI team is open to a contribution that adds dedicated support for vLLM.

Are there any existing plans or previous discussions around this? If not, I’d be happy to explore it further and share a prototype.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing Spring AI's existing OpenAI and Hugging Face integrations, then study the vLLM backend API and its compatibility with those abstractions. The issue does not name specific files or tests; done would require agreement on the integration scope and a maintainable native vLLM model implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.