Qwen3.5 使用官方脚本和数据集sft的时候loss为0

Checklist / 检查清单

• I have searched existing issues, and this is a new question or discussion topic. / 我已经搜索过现有的 issues，确认这是一个新的问题与讨论。

Question Description / 问题描述

PYTORCH_CUDA_ALLOC_CONF='expandable_segments:True'
NPROC_PER_NODE=4
MAX_PIXELS=1003520
VIDEO_MAX_PIXELS=50176
FPS_MAX_FRAMES=12
CUDA_VISIBLE_DEVICES=0,1,2,3
swift sft
--model Qwen/Qwen3.5-4B
--tuner_type lora
--dataset 'swift/self-cognition#500'
--load_from_cache_file true
--add_non_thinking_prefix true
--loss_scale ignore_empty_think
--split_dataset_ratio 0.01
--torch_dtype bfloat16
--num_train_epochs 1
--per_device_train_batch_size 4
--per_device_eval_batch_size 4
--learning_rate 1e-4
--lora_rank 8
--lora_alpha 32
--target_modules all-linear
--gradient_accumulation_steps 1
--group_by_length true
--output_dir output/Qwen3.5-4B
--eval_steps 50
--save_steps 50
--save_total_limit 2
--logging_steps 5
--max_length 2048
--warmup_ratio 0.05
--dataset_num_proc 4
--dataloader_num_workers 4
--deepspeed zero2
--model_author swift \

使用上述脚本在跑的时候出现loss为0的情况：
{'loss': '0', 'grad_norm': '1', 'learning_rate': '5e-05', 'token_acc': '0.0132', 'epoch': '0.03226', 'global_step/max_steps': '1/31', 'elapsed_time': '2s', 'remaining_time': '1m 6s', 'memory(GiB)': '10.26', 'train_speed(s/it)': '2.2'}
{'loss': '0', 'grad_norm': '1', 'learning_rate': '9.738e-05', 'token_acc': '0.0009935', 'epoch': '0.1613', 'global_step/max_steps': '5/31', 'elapsed_time': '4s', 'remaining_time': '23s', 'memory(GiB)': '10.29', 'train_speed(s/it)': '0.87'}
{'loss': '0', 'grad_norm': '1', 'learning_rate': '8.237e-05', 'token_acc': '0.005155', 'epoch': '0.3226', 'global_step/max_steps': '10/31', 'elapsed_time': '7s', 'remaining_time': '15s', 'memory(GiB)': '10.54', 'train_speed(s/it)': '0.6948'}
{'loss': '0', 'grad_norm': '1', 'learning_rate': '5.809e-05', 'token_acc': '0.005706', 'epoch': '0.4839', 'global_step/max_steps': '15/31', 'elapsed_time': '10s', 'remaining_time': '10s', 'memory(GiB)': '10.54', 'train_speed(s/it)': '0.6383'}
{'loss': '0', 'grad_norm': '1', 'learning_rate': '3.149e-05', 'token_acc': '0.00217', 'epoch': '0.6452', 'global_step/max_steps': '20/31', 'elapsed_time': '12s', 'remaining_time': '7s', 'memory(GiB)': '10.54', 'train_speed(s/it)': '0.6093'}
{'loss': '0', 'grad_norm': '1', 'learning_rate': '1.02e-05', 'token_acc': '0.007395', 'epoch': '0.8065', 'global_step/max_steps': '25/31', 'elapsed_time': '15s', 'remaining_time': '4s', 'memory(GiB)': '10.54', 'train_speed(s/it)': '0.589'}
{'loss': '0', 'grad_norm': '1', 'learning_rate': '2.9e-07', 'token_acc': '0.003868', 'epoch': '0.9677', 'global_step/max_steps': '30/31', 'elapsed_time': '17s', 'remaining_time': '1s', 'memory(GiB)': '10.54', 'train_speed(s/it)': '0.5744'}

不管是跑自己的数据集还是官方的，loss都是0，下面是环境重一些关键的包：

qwen-vl-utils 0.0.14
liger_kernel 0.7.0
causal_conv1d 1.6.1
flash_attn 2.8.3
flash-attn-3 3.0.0+20260318.cu128torch280cxx11abitrue.8afc61
flash-linear-attention 0.5.0
flashinfer-python 0.6.6
ms_swift 4.1.0.dev0 /efs/zarazywei/datasets/ms-swift
tokenizers 0.22.2
tomlkit 0.13.3
torch 2.10.0+cu129
torch_c_dlpack_ext 0.1.5
torchaudio 2.10.0
torchvision 0.25.0
tqdm 4.67.3
transformers 5.2.0
transformers-stream-generator 0.0.5
triton 3.6.0
trl 0.29.1
typer 0.24.1
typer-slim 0.24.0
typing_extensions 4.15.0
typing-inspection 0.4.2
tzdata 2025.3
urllib3 2.6.3
uvicorn 0.42.0
uvloop 0.22.1
vllm 0.18.0

是在H200上面跑的，python环境是12

Reproduce the reported swift sft command with the Qwen/Qwen3.5-4B model, swift/self-cognition#500 dataset, and the listed Python package versions on the H200 setup. Compare the logged loss, grad_norm, and token_acc across the official and custom datasets, then trace the training configuration to identify why loss is reported as zero and document a verified nonzero-loss result.