flagos-ai / flagos-ai/FlagTree
[FR] [Bug] vllm-plugin-FL fails to load with FlagTree on MUSA S5000: triton_v3.2.x reports 0 active drivers, main/triton 3.6 also fails
- Dominant language
- Python
- Stars
- 350
- Forks
- 149
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 81
Description
#[FR] # [Bug] vllm-plugin-FL fails to load with FlagTree on MUSA S5000: triton_v3.2.x reports 0 active drivers, main/triton 3.6 also fails
## Description
When using `vllm-plugin-FL` for FlagTree adaptation on Moore Threads / MUSA S5000, the vLLM service fails to start after installing FlagTree/Triton.
I tested both:
1. FlagTree / Triton 3.2 path
2. FlagTree main branch, which appears to be based on the Triton 3.6 path
Both fail to load the `fl` plugin.
For the Triton 3.2 path, the plugin fails during `flag_gems` import. The root cause seems to be that Triton cannot discover any active driver in the MUSA environment:
RuntimeError: 0 active drivers ([]). There should only be one.
After that, vLLM cannot infer the device type and exits:
RuntimeError: Failed to infer device type
# Steps to Reproduce
For the main branch / Triton 3.6 path, I installed FlagTree from source using the following general command sequence:
git clone https://github.com/flagos-ai/FlagTree.git
cd FlagTree
pip install . --no-build-isolation -v
export VLLM_PLUGINS=fl
export MUSA_VISIBLE_DEVICES=4,5,6,7
export VLLM_CONFIGURE_LOGGING=1
export VLLM_LOGGING_LEVEL=DEBUG
vllm serve /data/Qwen3.6-27B \
--served-model-name qwen \
--tensor-parallel-size 4 \
--host 0.0.0.0 \
--port 8099 \
--enforce-eager \
--trust-remote-code
The installation path also fails when starting vLLM with vllm-plugin-FL. Therefore this does not appear to be limited to only the 3.2 branch.
# Error Log
INFO 07-09 02:13:59 [__init__.py:44] Available plugins for group vllm.platform_plugins:
INFO 07-09 02:13:59 [__init__.py:46] - fl -> vllm_fl:register
INFO 07-09 02:13:59 [__init__.py:46] - musa -> vllm_musa:musa_platform_plugin
INFO 07-09 02:13:59 [__init__.py:58] Loading plugin fl
ERROR 07-09 02:14:00 [__init__.py:64] Failed to load plugin fl
The failure happens while importing flag_gems:
File "/workspace/vllm-plugin-FL/vllm_fl/__init__.py", line 20, in
from vllm_fl.utils import get_op_config as _get_op_config
File "/workspace/vllm-plugin-FL/vllm_fl/utils.py", line 7, in
import flag_gems
File "/usr/local/lib/python3.10/dist-packages/flag_gems/__init__.py", line 10, in
from flag_gems.fused import * # noqa: F403
File "/usr/local/lib/python3.10/dist-packages/flag_gems/fused/__init__.py", line 3, in
from flag_gems.fused.concat_and_cache_mla import concat_and_cache_mla
File "/usr/local/lib/python3.10/dist-packages/flag_gems/utils/shape_utils.py", line 265, in
def add_on_kernel(
File "/usr/local/lib/python3.10/dist-packages/triton/runtime/autotuner.py", line 130, in __init__
self.do_bench = driver.active.get_benchmarker()
File "/usr/local/lib/python3.10/dist-packages/triton/runtime/driver.py", line 8, in _create_driver
raise RuntimeError(f"{len(actives)} active drivers ({actives}). There should only be one.")
RuntimeError: 0 active drivers ([]). There should only be one.
Then vLLM exits with:
RuntimeError: Failed to infer device type, please set the environment variable `VLLM_LOGGING_LEVEL=DEBUG` to turn on verbose logging to help debug the issue.
# Environment
Component | Version
-- | --
Hardware | Moore Threads / MUSA / MTT S5000
GPU count | 8
GPU memory | 81920 MiB per card
Driver | 3.3.5-server
OS | Ubuntu 22.04.5 LTS
Architecture | x86_64
Python | 3.10
vLLM | 0.20.2+empty
vllm-plugin-FL | 0.2.0+gd1653d9ff
flag_gems | 5.0.0
PyTorch | 2.7.1
Original Triton in working container | 3.2.0
Original FlagTree in working container | None
Tested FlagTree / Triton paths | triton_v3.2.x / triton==3.2.0, and main branch / Triton 3.6 path
Model | Qwen3.6-27B
Tensor Parallel | 4
The original container environment uses vLLM 0.20.2+empty vllm-plugin-FL 0.2.0 FlagGems 5.0.0 Triton 3.2.0 and no FlagTree installed. After adding FlagTree or replacing Triton, the plugin fails to load.[harbor.baai.ac.cn/flagrelease-public/musa_gems_tree:2607081834](http://harbor.baai.ac.cn/flagrelease-public/musa_gems_tree:2607081834)
# Expected Behavior
The plugin should update its dependency to a version compatible with FlagTree's triton 3.6.0, or add compatibility handling for the missing function.flag_gemspow
# Reference
Plugin adaptation documentation: https://qcnje6z2ldat.feishu.cn/docx/SEHtdWS3Go20Qbx4K30cI7JPnBf
Contributor guide
Research direction
Start by reproducing the failure with the listed FlagTree/Triton 3.2 and main/Triton 3.6 paths. Read vllm_fl/__init__.py and vllm_fl/utils.py, then trace the flag_gems import into Triton's runtime/driver.py where zero active drivers are reported. Done means the fl plugin loads successfully in the affected MUSA environment and vLLM can infer the device type.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100