vllm-project / vllm-project/aibrix

[Feature]: Integrate StormService Placement with vLLM-Omni Stage Topology

Open
#2,009 0 comments 1 reaction 1 assignee Claimed by @Lucas-Qian6 View on GitHub
Dominant language
Go
Stars
5.1k
Forks
694
Avg merge
1d 20h
Merged PRs (30d)
98

Description

### Summary

Investigate and design how AIBrix StormService should integrate with vLLM-Omni for multi-stage model serving, where StormService manages pod/role/GPU placement and vLLM-Omni manages stage topology and inter-stage communication.

The goal is to define the integration boundary and identify the adapter layer needed to map StormService deployment decisions into vLLM-Omni stage configuration and launch parameters.

### Motivation

Vanilla vLLM-Omni already supports multi-stage distributed serving through its own stage YAML, launch arguments, and connector system such as Mooncake. In that setup, the global stage topology is defined directly by vLLM-Omni.

In AIBrix, however, StormService becomes the pod-level orchestrator. It manages:
• role and pod placement
• GPU allocation per pod
• replica management
• autoscaling and lifecycle

This introduces a second deployment layer on top of vLLM-Omni’s own stage-topology model.

The key problem is: How should StormService’s pod/role/resource decisions be translated into the stage topology and runtime configuration expected by vLLM-Omni?

### Proposed Change

A new adapter layer should:
• map StormService role/pod layout to vLLM-Omni stage layout
• generate or select the correct vLLM-Omni stage YAML
• inject required runtime parameters such as: stage IDs, stage subset for each pod, peer stage addresses / service DNS and Mooncake-related connector configuration

### Alternatives Considered

_No response_

### Related Issue

#1966

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.