Qwen3.5 视觉任务推理报错：CUDA error: an illegal memory access was encountered

Checklist / 检查清单

• I have searched existing issues, and this is a new bug report. / 我已经搜索过现有的 issues，确认这是一个新的 bug report。

Bug Description / Bug 描述

训练正常，推理报错：
报错信息如下：

  File "/data/dongyongfei/venv/dyf-py312/lib/python3.12/site-packages/flash_attn/flash_attn_interface.py", line 165, in _flash_attn_varlen_forward
    out, softmax_lse, S_dmask, rng_state = flash_attn_gpu.varlen_fwd(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.AcceleratorError: CUDA error: an illegal memory access was encountered

How to Reproduce / 如何复现

CUDA_VISIBLE_DEVICES=0 swift infer \
    --model_type qwen3_5 \
    --adapters xxx \
    --infer_backend pt \
    --max_batch_size 1 \
    --max_new_tokens 512\
    --max_model_len 80000\
    --temperature 0 \
    --val_dataset xxx  \
    --result_path xxx

主要依赖包：

torch                         2.9.0
torchvision                   0.24.0
transformer_engine            2.13.0
transformers                  5.3.0
ms_swift                      4.1.0.dev0   
flash_attn                    2.8.3
flash-linear-attention        0.4.2

Additional Information / 补充信息

transformers库里面有类似issue：https://github.com/huggingface/transformers/issues/44910
No response

Start by reproducing the provided swift infer command with the actual model, adapter, dataset, and result paths, then inspect the failing flash_attn_interface.py _flash_attn_varlen_forward call. Compare the installed PyTorch, Transformers, FlashAttention, and ms-swift versions with the linked Transformers issue. Done means Qwen3.5 visual inference completes without the illegal memory-access error.