vllm-project / vllm-project/production-stack

feature: add first-class multi-node multiprocessing deployment support

Open
#930 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.6k
Forks
503
Avg merge
4d 17h
Merged PRs (30d)
8

Description

Describe the feature

Please add first-class support for multi-node multiprocessing deployment in production-stack.

Right now, the distributed Kubernetes path is centered around the RayCluster template:
https://github.com/vllm-project/production-stack/blob/main/helm/templates/ray-cluster.yaml

That template currently hardcodes Ray-based execution, while recent vLLM changes indicate that multi-node deployment should also support the multiprocessing path:
https://github.com/vllm-project/vllm/pull/39400

It would be useful if production-stack exposed a supported way to configure multi-node MP deployment parameters such as nnodes, node-rank, master-addr, master-port, and headless.

Why do you need this feature?

I am trying to deploy a single distributed model across 4 Kubernetes nodes with production-stack.

At the moment, the chart supports the Ray-based path, but does not provide a first-class way to deploy multi-node vLLM with multiprocessing. That makes it difficult to follow the newer vLLM deployment direction without maintaining custom manifests outside of production-stack.

Additional context

Concrete improvements that would help:

  • avoid hardcoding Ray-only execution in the current distributed template
  • expose multi-node multiprocessing settings through chart values
  • provide an official Kubernetes example for multi-node MP deployment

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with helm/templates/ray-cluster.yaml and review the linked vLLM multiprocessing changes to understand the required multi-node parameters. Trace how the chart currently exposes deployment values, then define the configuration and official Kubernetes example needed for nnodes, node-rank, master-addr, master-port, and headless. Done means the chart supports a documented multi-node multiprocessing deployment without custom manifests.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes
Domain
devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.