单机多卡加载模型时卡住

torch 1.13
cuda 11.7

推理代码能正常运行

训练开4卡4090，加载模型时卡住，cpu占用100%，显卡占用100%

torchrun --nnodes=1 --nproc_per_node=4 --rdzv_id=1 --rdzv_backend=c10d --rdzv_endpoint=localhost:12345 finetune_cpm_bee.py --use-delta --model-config config/cpm-bee-10b.json --dataset datasets/eprstmt/binary/dev --eval_dataset datasets/eprstmt/binary/eval_dev --epoch 100 --batch-size 4 --train-iters 100 --save-name cpm_bee_finetune --max-length 2048 --save results/ --lr 0.0001 --inspect-iters 100 --warmup-iters 1 --eval-interval 1000 --early-stop-patience 5 --lr-decay-style noam --weight-decay 0.01 --clip-grad 1.0 --loss-scale 32768 --start-step 0 --load path/pytorch_model_10b.bin

====================== Initialization ======================
rank : 0
local_rank : 0
world_size : 4
local_size : 4
master : star-SYS-420GP-TNR:37257
device : 0
cpus : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 1
3, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 2
4, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 3
5, 36, 37]

Start by reproducing the four-GPU command with torchrun and the checkpoint path shown in the issue, then inspect finetune_cpm_bee.py and config/cpm-bee-10b.json around model loading and distributed initialization. Compare rank and device behavior across processes. Done means the model loads and training begins on all four GPUs without the reported CPU/GPU stall.