NVIDIA-NeMo / NVIDIA-NeMo/RL

Qwen 3.5 with Megatron-inference

Open
#3,010 1 comment 0 reactions 0 assignees View on GitHub
bug community-request waiting-on-maintainers
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

Meet error when running Qwen 3.5 RL with Megatron-inference and sglang. More details please check the attached log.

model: Qwen3.5 35B A3B VLM
params: use cuda graph with Megatron-inference
Question: which model does Megatron-inference validate?

[qwen3.5_minf_error.log](https://github.com/user-attachments/files/29499710/qwen3.5_minf_error.log)

This problem seems to be the issue of cuda graph.
But when I disable the cuda graph, it will hang with Qwen3.5 + Megatron inference FLA kernel.

Contributor guide

Open the contributing guide

Research direction

Start by reading the attached qwen3.5_minf_error.log and reproducing the Qwen3.5 35B A3B VLM RL run with Megatron-inference and sglang. Compare behavior with CUDA graphs enabled and disabled, including the reported FLA-kernel hang; done means identifying the failing path and confirming which model Megatron-inference validates.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.