Wan2.1-VACE-1.3B training issues

Although the VACE-1.3B model is a bit outdated, I'm still puzzled by the issue below.

Here's the script:

CUDA_VISIBLE_DEVICES=6,3 accelerate launch --num_processes 2 examples/wanvideo/model_training/train.py --dataset_base_path data/example_video_dataset --data_file_keys "video" --height 480 --width 832 --num_frames 21 --dataset_repeat 100 --learning_rate 1e-4 --num_epochs 1 --output_path ./models/train/Wan2.1-VACE-1.3B_full --trainable_models "vace" --use_gradient_checkpointing_offload --remove_prefix_in_ckpt "pipe.vace" --model_paths '["/data/HuggingFace/VACE/Wan2.1-VACE-1.3B/diffusion_pytorch_model.safetensors","/data/HuggingFace/VACE/Wan2.1-VACE-1.3B/models_t5_umt5-xxl-enc-bf16.pth","/data/HuggingFace/VACE/Wan2.1-VACE-1.3B/Wan2.1_VAE.pth"]'

Here's the error info:

[rank1]: Traceback (most recent call last): [rank1]: File "/data/liyifan/clone/DiffSynth-Studio/examples/wanvideo/model_training/train.py", line 126, in <module> [rank1]: launch_training_task( [rank1]: File "/data/liyifan/clone/DiffSynth-Studio/diffsynth/trainers/utils.py", line 422, in launch_training_task [rank1]: accelerator.backward(loss) [rank1]: File "/home/liyifan/miniforge3/envs/diffsynth/lib/python3.10/site-packages/accelerate/accelerator.py", line 2578, in backward [rank1]: loss.backward(**kwargs) [rank1]: File "/home/liyifan/miniforge3/envs/diffsynth/lib/python3.10/site-packages/torch/_tensor.py", line 648, in backward [rank1]: torch.autograd.backward( [rank1]: File "/home/liyifan/miniforge3/envs/diffsynth/lib/python3.10/site-packages/torch/autograd/__init__.py", line 353, in backward [rank1]: _engine_run_backward( [rank1]: File "/home/liyifan/miniforge3/envs/diffsynth/lib/python3.10/site-packages/torch/autograd/graph.py", line 824, in _engine_run_backward [rank1]: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass [rank1]: RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

there was a similar issue but it didn't work for me Wan2.1-VACE-14B.sh script issue

could anyone give me some suggestion :)

Start with examples/wanvideo/model_training/train.py and trace the call into diffsynth/trainers/utils.py around launch_training_task and accelerator.backward(loss). Run the supplied accelerate command, then compare its behavior with the related Wan2.1-VACE-14B issue (#703). Done means the reported training path reaches backward without the no-grad error, with the cause and required change documented.