vllm-project / vllm-project/production-stack

feature: Helm Chart configs for fallback CPU support

Open
#356 0 comments 5 reactions 1 assignee View on GitHub

Nobody has claimed this yet.

feature request
Dominant language
Python
Stars
2.6k
Forks
503
Avg merge
4d 17h
Merged PRs (30d)
8

Description

Describe the feature

Enable dynamic switching between CPU and GPU configurations with vLLM Helm charts based on device availability. This feature would simplify the transition from local development environments (CPU-only) to production environments (GPU-enabled), improving flexibility and scalability.

Currently, the vLLM Helm charts assume GPU availability for deployment, as seen in the values.yaml configurations (resources.limits.nvidia.com/gpu). Users without GPU resources face challenges adapting the charts for CPU-only environments. The current workaround involves manually modifying container images or resource configurations, which is cumbersome and error-prone.

Why do you need this feature?
  • Simplifies deployment for users without GPUs.
  • Reduces manual intervention during environment transitions.
  • Enhances usability of vLLM Helm charts across diverse Kubernetes setups.
  • Encourages broader adoption by accommodating resource-constrained environments.
Additional context

With no GPU nodes available, I'm struggling with local development of a basic embedding service within a Minikube stack. I'm willing to contribute a new tutorial if anyone os willing to point me in the right direction on this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.