vllm-project / vllm-project/aibrix
[Feature]: Integrate StormService Placement with vLLM-Omni Stage Topology
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 98
Description
### Summary
Investigate and design how AIBrix StormService should integrate with vLLM-Omni for multi-stage model serving, where StormService manages pod/role/GPU placement and vLLM-Omni manages stage topology and inter-stage communication.
The goal is to define the integration boundary and identify the adapter layer needed to map StormService deployment decisions into vLLM-Omni stage configuration and launch parameters.
### Motivation
Vanilla vLLM-Omni already supports multi-stage distributed serving through its own stage YAML, launch arguments, and connector system such as Mooncake. In that setup, the global stage topology is defined directly by vLLM-Omni.
In AIBrix, however, StormService becomes the pod-level orchestrator. It manages:
• role and pod placement
• GPU allocation per pod
• replica management
• autoscaling and lifecycle
This introduces a second deployment layer on top of vLLM-Omni’s own stage-topology model.
The key problem is: How should StormService’s pod/role/resource decisions be translated into the stage topology and runtime configuration expected by vLLM-Omni?
### Proposed Change
A new adapter layer should:
• map StormService role/pod layout to vLLM-Omni stage layout
• generate or select the correct vLLM-Omni stage YAML
• inject required runtime parameters such as: stage IDs, stage subset for each pod, peer stage addresses / service DNS and Mooncake-related connector configuration
### Alternatives Considered
_No response_
### Related Issue
#1966
Contributor guide
Assessment
This issue has not been assessed yet.