vllm-project / vllm-project/aibrix

Request: Create a Service for the Pod with POD_GROUP_INDEX=0 in Distributed P/D Deployment

Open
#1,760 4 comments 0 reactions 1 assignee Claimed by @googs1025 View on GitHub
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

Hi,

I am trying to deploy a vLLM application with P/D disaggregation, where both the P and D components run in distributed mode. In this setup, we rely on Ray to form clusters for both P and D.

During testing, I noticed that for either P or D, the head node (selected as the pod with POD_GROUP_INDEX=0) does not get a corresponding Service created. Since the head pod requires a stable Service endpoint for Ray cluster initialization, this causes issues in forming the cluster.

**Request:**
Would the community consider **automatically creating a Service** for the pod with POD_GROUP_INDEX=0 in each P/D group when running in distributed mode?

This would simplify cluster formation and ensure Ray can properly bootstrap the head node.

Thank you!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.