modelscope / modelscope/DiffSynth-Studio
train with Wan2.2_I2V_A14B will encounter the Wan2.1-T2V-1.3B, although the Wan2.2 local ckpts paths are given.
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
pretty curious.
#!/bin/bash
# set -ex
accelerate launch \
--main_process_ip $MASTER_ADDR \
--main_process_port $MASTER_PORT \
--machine_rank $RANK \
--num_machines $WORLD_SIZE \
--num_processes $WORLD_SIZE \
--multi_gpu \
--mixed_precision bf16 \
--num_cpu_threads_per_process 1 \
examples/wanvideo/model_training/train.py \
--dataset_base_path /home/work/data/ \
--dataset_metadata_path /home/work/data/metadata.csv \
--height 544 \
--width 960 \
--num_frames 37 \
--dataset_repeat 100 \
--model_paths '[
[
"/home/work/DiffSynth-Studio/Wan-AI/Wan2.2-I2V-A14B/high_noise_model/diffusion_pytorch_model-00001-of-00006.safetensors",
"/home/work/DiffSynth-Studio/Wan-AI/Wan2.2-I2V-A14B/high_noise_model/diffusion_pytorch_model-00002-of-00006.safetensors",
"/home/work/DiffSynth-Studio/Wan-AI/Wan2.2-I2V-A14B/high_noise_model/diffusion_pytorch_model-00003-of-00006.safetensors",
"/home/work/DiffSynth-Studio/Wan-AI/Wan2.2-I2V-A14B/high_noise_model/diffusion_pytorch_model-00004-of-00006.safetensors",
"/home/work/DiffSynth-Studio/Wan-AI/Wan2.2-I2V-A14B/high_noise_model/diffusion_pytorch_model-00005-of-00006.safetensors",
"/home/work/DiffSynth-Studio/Wan-AI/Wan2.2-I2V-A14B/high_noise_model/diffusion_pytorch_model-00006-of-00006.safetensors"
],
"/home/work/DiffSynth-Studio/Wan-AI/Wan2.2-I2V-A14B/models_t5_umt5-xxl-enc-bf16.pth",
"/home/work/DiffSynth-Studio/Wan-AI/Wan2.2-I2V-A14B/Wan2.1_VAE.pth"
]' \
--learning_rate 1e-4 \
--num_epochs 5 \
--remove_prefix_in_ckpt "pipe.dit." \
--output_path "/home/work/data_OUTPUT/Wan2.2-I2V-A14B_high_noise_lora" \
--lora_base_model "dit" \
--lora_target_modules "q,k,v,o,ffn.0,ffn.2" \
--lora_rank 32 \
--extra_inputs "input_image" \
--max_timestep_boundary 0.358 \
--min_timestep_boundary 0
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/wanvideo/model_training/train.py and reproduce the provided accelerate command with the Wan2.2-I2V-A14B paths. Trace how --model_paths is parsed and which checkpoint is selected; done means training uses the supplied Wan2.2 model rather than Wan2.1-T2V-1.3B.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100