Qwen 3.5 with Megatron-inference
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
Meet error when running Qwen 3.5 RL with Megatron-inference and sglang. More details please check the attached log.
model: Qwen3.5 35B A3B VLM
params: use cuda graph with Megatron-inference
Question: which model does Megatron-inference validate?
[qwen3.5_minf_error.log](https://github.com/user-attachments/files/29499710/qwen3.5_minf_error.log)
This problem seems to be the issue of cuda graph.
But when I disable the cuda graph, it will hang with Qwen3.5 + Megatron inference FLA kernel.
Contributor guide
Research direction
Start by reading the attached qwen3.5_minf_error.log and reproducing the Qwen3.5 35B A3B VLM RL run with Megatron-inference and sglang. Compare behavior with CUDA graphs enabled and disabled, including the reported FLA-kernel hang; done means identifying the failing path and confirming which model Megatron-inference validates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100