vllm-project / vllm-project/aibrix
Request: Create a Service for the Pod with POD_GROUP_INDEX=0 in Distributed P/D Deployment
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
Hi,
I am trying to deploy a vLLM application with P/D disaggregation, where both the P and D components run in distributed mode. In this setup, we rely on Ray to form clusters for both P and D.
During testing, I noticed that for either P or D, the head node (selected as the pod with POD_GROUP_INDEX=0) does not get a corresponding Service created. Since the head pod requires a stable Service endpoint for Ray cluster initialization, this causes issues in forming the cluster.
**Request:**
Would the community consider **automatically creating a Service** for the pod with POD_GROUP_INDEX=0 in each P/D group when running in distributed mode?
This would simplify cluster formation and ensure Ray can properly bootstrap the head node.
Thank you!
Contributor guide
Assessment
This issue has not been assessed yet.