vllm-project / vllm-project/production-stack
feature: Helm Chart configs for fallback CPU support
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 503
- Avg merge
- 4d 17h
- Merged PRs (30d)
- 8
Description
Describe the feature
Enable dynamic switching between CPU and GPU configurations with vLLM Helm charts based on device availability. This feature would simplify the transition from local development environments (CPU-only) to production environments (GPU-enabled), improving flexibility and scalability.
Currently, the vLLM Helm charts assume GPU availability for deployment, as seen in the values.yaml configurations (resources.limits.nvidia.com/gpu). Users without GPU resources face challenges adapting the charts for CPU-only environments. The current workaround involves manually modifying container images or resource configurations, which is cumbersome and error-prone.
Why do you need this feature?
- Simplifies deployment for users without GPUs.
- Reduces manual intervention during environment transitions.
- Enhances usability of vLLM Helm charts across diverse Kubernetes setups.
- Encourages broader adoption by accommodating resource-constrained environments.
Additional context
With no GPU nodes available, I'm struggling with local development of a basic embedding service within a Minikube stack. I'm willing to contribute a new tutorial if anyone os willing to point me in the right direction on this.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.