vllm-project / vllm-project/production-stack
feature: Support CRD based configuration
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 503
- Avg merge
- 4d 17h
- Merged PRs (30d)
- 8
Description
Describe the feature
I understand that if i use the router in a kubernetes environment, the router uses service discovery to build a pool of backend vllm services. While this worry-free mechanism is good in many ways, i am wondering if we can add yet another way to explicitly declare a backend connectivity through a kubernetes crd. The idea is that, for certain use cases, the crd represents a persistent configuration that can be managed by controllers (while annotation and labels that are fed to service discovery are mostly ad hoc)
Why do you need this feature?
Support routing CRD so that a declarative management can be supported instead of using service discovery.
Additional context
(see the slack discussion https://vllm-dev.slack.com/archives/C089SMEAKRA/p1740756084674449)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue mentions the router, Kubernetes service discovery, annotations and labels, but names no files or tests. Start by locating the router's service-discovery integration and the Kubernetes configuration entry points, then determine the CRD shape and how it should represent backend connectivity. Done means the requested declarative CRD-based routing behavior is specified and implemented with coverage for the relevant integration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- backend-api-design, cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100