[Bug] qwen vl broken with qwen35 megatron bridge
Open
Nobody has claimed this yet.
bug
- Dominant language
- Python
- Stars
- 8.5k
- Forks
- 1.3k
- Avg merge
- 5h 36m
- Merged PRs (30d)
- 22
Description
Bug Description
(SGLangEngine pid=87820) File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler_update_weights_mixin.py", line 95, in update_weights_from_tensor [repeated 6x across cluster]
(SGLangEngine pid=87820) success, message = worker.update_weights_from_tensor(recv_req) [repeated 6x across cluster]
(SGLangEngine pid=87820) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 12x across cluster]
(SGLangEngine pid=87820) File "/sgl-workspace/sglang/python/sglang/srt/managers/tp_worker.py", line 159, in update_weights_from_tensor [repeated 6x across cluster]
(SGLangEngine pid=87820) success, message = self.model_runner.update_weights_from_tensor( [repeated 6x across cluster]
(SGLangEngine pid=87820) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster]
(SGLangEngine pid=87820) File "/sgl-workspace/sglang/python/sglang/srt/model_executor/model_runner.py", line 1437, in update_weights_from_tensor [repeated 6x across cluster]
(SGLangEngine pid=87820) return self._update_weights_from_flattened_bucket( [repeated 6x across cluster]
(SGLangEngine pid=87820) File "/sgl-workspace/sglang/python/sglang/srt/model_executor/model_runner.py", line 1488, in _update_weights_from_flattened_bucket [repeated 6x across cluster]
(SGLangEngine pid=87820) self.model.load_weights(reconstructed_tensors) [repeated 6x across cluster]
(SGLangEngine pid=87820) File "/sgl-workspace/sglang/python/sglang/srt/models/qwen3_vl_moe.py", line 288, in load_weights [repeated 6x across cluster]
(SGLangEngine pid=87820) load_fused_expert_weights( [repeated 6x across cluster]
(SGLangEngine pid=87820) File "/sgl-workspace/sglang/python/sglang/srt/models/qwen3_vl_moe.py", line 162, in load_fused_expert_weights [repeated 6x across cluster]
(SGLangEngine pid=87820) weight_loader( [repeated 6x across cluster]
(SGLangEngine pid=87820) File "/sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/layer.py", line 633, in weight_loader [repeated 6x across cluster]
(SGLangEngine pid=87820) self._weight_loader_physical( [repeated 6x across cluster]
(SGLangEngine pid=87820) File "/sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/layer.py", line 663, in _weight_loader_physical [repeated 6x across cluster]
(SGLangEngine pid=87820) self._weight_loader_impl( [repeated 6x across cluster]
(SGLangEngine pid=87820) File "/sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/layer.py", line 859, in _weight_loader_impl [repeated 6x across cluster]
(SGLangEngine pid=87820) self._load_model_weight_or_group_weight_scale( [repeated 6x across cluster]
(SGLangEngine pid=87820) File "/sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/layer.py", line 367, in _load_model_weight_or_group_weight_scale [repeated 6x across cluster]
(SGLangEngine pid=87820) File "/sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/layer.py", line 459, in _load_w13 [repeated 6x across cluster]
(SGLangEngine pid=87820) expert_data.copy_(loaded_weight) [repeated 6x across cluster]
(SGLangEngine pid=87820) RuntimeError: The size of tensor a (2048) must match the size of tensor b (1536) at non-singleton dimension 1 [repeated 6x across cluster]
(SGLangEngine pid=87820) [2026-03-13 05:17:06] SIGQUIT received. signum=None, frame=None. It usually means one child failed.
Steps to Reproduce
When using https://github.com/coding-famer/Megatron-Bridge-slime/tree/qwen35, qwen3 vl is broken.
with image dataset
Expected Behavior
no
Actual Behavior
no
Environment
- slime version:
- Python version:
- PyTorch version:
- CUDA/ROCm version:
- GPU type and count:
- OS:
- SGLang version (if relevant):
- Megatron-LM version (if relevant):
Logs
Additional Context
No response
Pre-submission Checklist
- I have read the CONTRIBUTING.md and understand the collaboration scope.
- I have read the documentation and my issue is not addressed there.
- I have searched for existing issues and this is not a duplicate.
- I have provided a minimal, reproducible example.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with python/sglang/srt/models/qwen3_vl_moe.py at load_weights and load_fused_expert_weights, then trace the dimension handling into layers/moe/fused_moe_triton/layer.py at _load_w13. Reproduce the qwen3 VL image-dataset path using the linked Megatron Bridge branch and compare the reconstructed tensor dimensions; done means weight loading completes without the reported mismatch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100