Support use vLLM DP + EP in async engine
Open
Feature
Speed
vllm
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
https://github.com/NVIDIA-NeMo/RL/pull/1081 supports using vLLM DP + EP in sync engine.
vLLM's DP + EP is supported for async engine for serve mode, but has issues for offline mode (which we used).
We need to hack something in vLLM to support that.
Ideally, after vLLM's FP8 inference for MoE model and DeepEP supported, we can get some speedup from DP + EP, and we need to bring the benefit to async RL / async rollout which needs to use vLLM async engine.
Contributor guide
Assessment
This issue has not been assessed yet.