modelscope / modelscope/ms-swift

华为昇腾训练kto数据集时loss不下降

Open
#6,536 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

npu stale
Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

Describe the bug
What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程,最好有截图)
在两张华为昇腾910b2上使用kto数据集训练qwen2.5-32b-Instruct模型时,loss并没有下降,但是在A100显卡上训练时,loss能正常下降。
下面是在910b2上的日志截图

Image 下面是A100上的日志截图 Image

GPU和npu两边训练环境是一致的,同样都是两卡,执行命令也是一致的
执行命令如下:
ASCEND_RT_VISIBLE_DEVICES=0,1 NPROC_PER_NODE=2 swift rlhf --rlhf_type kto --model /app/Qwen2.5-32B-Instruct --train_type lora --dataset /app/ms-swift-3.9.0/merged_kto_data_final.json --split_dataset_ratio 0.01 --num_train_epochs 1 --torch_dtype bfloat16 --per_device_train_batch_size 4 --per_device_eval_batch_size 1 --learning_rate 1e-4 --lora_rank 8 --lora_alpha 32 --target_modules all-linear --gradient_accumulation_steps 8 --eval_steps 100 --save_steps 100 --save_total_limit 2 --logging_steps 5 --max_length 2048 --output_dir output --warmup_ratio 0.1 --dataloader_num_workers 16 --deepspeed zero3 --dataset_num_proc 16

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reported swift rlhf KTO command and reproduce the run on two Ascend 910B2 devices, comparing its logs with the A100 run. Trace the KTO training path and NPU-specific behavior; done means identifying why the Ascend loss remains flat and confirming that the same setup produces the expected decreasing loss.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.