qwen3vl-8b训练出现错误：ValueError('numel (2097152) exceeds triton maximum tensor numel (1048576)')

Checklist / 检查清单

• I have searched existing issues, and this is a new bug report. / 我已经搜索过现有的 issues，确认这是一个新的 bug report。

Bug Description / Bug 描述

有人在训练qwen3-vl-8b的时候开启--use_liger_kernel true 遇到这个错误嘛
要如何解决呢

[rank1]: Traceback (most recent call last):
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/cli/sft.py", line 20, in <module>
[rank1]:     sft_main()
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/llm/train/sft.py", line 365, in sft_main
[rank1]:     return SwiftSft(args).main()
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/llm/base.py", line 49, in main
[rank1]:     result = self.run()
[rank1]:              ^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/ray/base.py", line 170, in wrapper
[rank1]:     return func(self, *args, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/llm/train/sft.py", line 210, in run
[rank1]:     return self.train(trainer)
[rank1]:            ^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/llm/train/sft.py", line 258, in train
[rank1]:     trainer.train(trainer.args.resume_from_checkpoint)
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/trainers/mixin.py", line 880, in train
[rank1]:     res = super().train(*args, **kwargs)
[rank1]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/transformers/trainer.py", line 2325, in train
[rank1]:     return inner_training_loop(
[rank1]:            ^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/transformers/trainer.py", line 2674, in _inner_training_loop
[rank1]:     tr_loss_step = self.training_step(model, inputs, num_items_in_batch)
[rank1]:                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/trainers/trainers.py", line 443, in training_step
[rank1]:     return super().training_step(model, inputs, *args, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/transformers/trainer.py", line 4020, in training_step
[rank1]:     loss = self.compute_loss(model, inputs, num_items_in_batch=num_items_in_batch)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/trainers/trainers.py", line 353, in compute_loss
[rank1]:     outputs = model(**inputs)
[rank1]:               ^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
[rank1]:     return self._call_impl(*args, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
[rank1]:     return forward_call(*args, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/deepspeed/utils/nvtx.py", line 20, in wrapped_fn
[rank1]:     ret_val = func(*args, **kwargs)
[rank1]:               ^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/deepspeed/runtime/engine.py", line 2054, in forward
[rank1]:     loss = self.module(*inputs, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
[rank1]:     return self._call_impl(*args, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1881, in _call_impl
[rank1]:     return inner()
[rank1]:            ^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1829, in inner
[rank1]:     result = forward_call(*args, **kwargs)
[rank1]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/transformers/utils/generic.py", line 918, in wrapper
[rank1]:     output = func(self, *args, **kwargs)
[rank1]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/liger_kernel/transformers/model/qwen3_vl.py", line 87, in lce_forward
[rank1]:     outputs = self.model(
[rank1]:               ^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
[rank1]:     return self._call_impl(*args, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1881, in _call_impl
[rank1]:     return inner()
[rank1]:            ^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1829, in inner
[rank1]:     result = forward_call(*args, **kwargs)
[rank1]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/transformers/utils/generic.py", line 1072, in wrapper
[rank1]:     outputs = func(self, *args, **kwargs)
[rank1]:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/llm/model/model/qwen.py", line 1039, in forward
[rank1]:     inputs_embeds, visual_pos_masks, deepstack_visual_embeds = _forward_qwen3_vl_or_qwen3_omni(
[rank1]:                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/llm/model/model/qwen.py", line 936, in _forward_qwen3_vl_or_qwen3_omni
[rank1]:     mixed_embeds, deepstack_visual_embeds = self.visual(pixel_values_mixed, grid_thw=grid_thw)
[rank1]:                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
[rank1]:     return self._call_impl(*args, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1881, in _call_impl
[rank1]:     return inner()
[rank1]:            ^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1829, in inner
[rank1]:     result = forward_call(*args, **kwargs)
[rank1]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/transformers/models/qwen3_vl/modeling_qwen3_vl.py", line 739, in forward
[rank1]:     hidden_states = blk(
[rank1]:                     ^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/transformers/modeling_layers.py", line 93, in __call__
[rank1]:     return self._gradient_checkpointing_func(partial(super().__call__, **kwargs), *args)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/swift/260118/ms-swift-main/swift/trainers/mixin.py", line 812, in _new_checkpoint
[rank1]:     return _old_checkpoint(*args, use_reentrant=use_reentrant_, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/_compile.py", line 53, in inner
[rank1]:     return disable_fn(*args, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/_dynamo/eval_frame.py", line 1044, in _fn
[rank1]:     return fn(*args, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/utils/checkpoint.py", line 496, in checkpoint
[rank1]:     return CheckpointFunction.apply(function, preserve, *args)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/autograd/function.py", line 581, in apply
[rank1]:     return super().apply(*args, **kwargs)  # type: ignore[misc]
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/utils/checkpoint.py", line 262, in forward
[rank1]:     outputs = run_function(*args)
[rank1]:               ^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
[rank1]:     return self._call_impl(*args, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1881, in _call_impl
[rank1]:     return inner()
[rank1]:            ^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1829, in inner
[rank1]:     result = forward_call(*args, **kwargs)
[rank1]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/transformers/models/qwen3_vl/modeling_qwen3_vl.py", line 267, in forward
[rank1]:     hidden_states = hidden_states + self.attn(
[rank1]:                                     ^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
[rank1]:     return self._call_impl(*args, **kwargs)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1881, in _call_impl
[rank1]:     return inner()
[rank1]:            ^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1829, in inner
[rank1]:     result = forward_call(*args, **kwargs)
[rank1]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/transformers/models/qwen3_vl/modeling_qwen3_vl.py", line 195, in forward
[rank1]:     query_states, key_states = apply_rotary_pos_emb_vision(query_states, key_states, cos, sin)
[rank1]:                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/liger_kernel/transformers/rope.py", line 62, in liger_rotary_pos_emb_with_cast_and_leading_batch
[rank1]:     q_out, k_out = liger_rotary_pos_emb(q32, k32, cos32, sin32, position_ids=position_ids, unsqueeze_dim=unsqueeze_dim)
[rank1]:                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/liger_kernel/transformers/rope.py", line 25, in liger_rotary_pos_emb
[rank1]:     return LigerRopeFunction.apply(q, k, cos, sin, position_ids, unsqueeze_dim)
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/torch/autograd/function.py", line 581, in apply
[rank1]:     return super().apply(*args, **kwargs)  # type: ignore[misc]
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/liger_kernel/ops/rope.py", line 225, in forward
[rank1]:     q, k, cos, sin = rope_forward(q, k, cos, sin)
[rank1]:                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/liger_kernel/ops/rope.py", line 137, in rope_forward
[rank1]:     _triton_rope[(n_row,)](
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/triton/runtime/jit.py", line 419, in <lambda>
[rank1]:     return lambda *args, **kwargs: self.run(grid=grid, warmup=False, *args, **kwargs)
[rank1]:                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/triton/runtime/jit.py", line 733, in run
[rank1]:     kernel = self._do_compile(key, signature, device, constexprs, options, attrs, warmup)
[rank1]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/triton/runtime/jit.py", line 861, in _do_compile
[rank1]:     kernel = self.compile(src, target=target, options=options.__dict__)
[rank1]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/triton/compiler/compiler.py", line 300, in compile
[rank1]:     module = src.make_ir(target, options, codegen_fns, module_map, context)
[rank1]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]:   File "/Ds/xmu/linjie/conda_env/swift-sft/swift-sft/lib/python3.12/site-packages/triton/compiler/compiler.py", line 80, in make_ir
[rank1]:     return ast_to_ttir(self.fn, self, context=context, options=options, codegen_fns=codegen_fns,
[rank1]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: triton.compiler.errors.CompilationError: at 69:27:

[rank1]:     cos_offsets = tl.arange(0, pad_hd // 2)
[rank1]:     cos_mask = cos_offsets < hd // 2
[rank1]:     cos_row = tl.load(cos + cos_offsets, mask=cos_mask, other=0)
[rank1]:     sin_row = tl.load(sin + cos_offsets, mask=cos_mask, other=0)

[rank1]:     # ####################################################################
[rank1]:     # Load the left and right half of q and k for the current
[rank1]:     # program instance (i.e. for the current token) separately
[rank1]:     # ####################################################################
[rank1]:     # left half of the head
[rank1]:     first_half_q_offsets = tl.arange(0, pad_n_qh)[:, None] * hd + tl.arange(0, pad_hd // 2)[None, :]
[rank1]:                            ^
[rank1]: ValueError('numel (2097152) exceeds triton maximum tensor numel (1048576)')

How to Reproduce / 如何复现

conda create --name swift-sft python==3.12 -y

conda install -c "nvidia/label/cuda-12.6.0" cuda-toolkit -y

git clone https://github.com/modelscope/ms-swift.git
cd ms-swift/
pip install -e '.[all]' --index-url https://pypi.tuna.tsinghua.edu.cn/simple/

flash-attn : https://github.com/Dao-AILab/flash-attention/releases

pip install liger-kernel
pip install "math_verify==0.5.2"
pip install "deepspeed<0.17" -U

使用的是torch2.9.1

Additional Information / 补充信息

No response

Start by reproducing Qwen3-VL-8B SFT with --use_liger_kernel true and inspect the traceback around swift/llm/model/model/qwen.py, especially _forward_qwen3_vl_or_qwen3_omni. Compare the Qwen3-VL vision rotary-position path with the Liger rope implementation named in the traceback. Done means the documented training configuration runs without the tensor-numel error, with a regression test or confirmed compatibility evidence.