modelscope / modelscope/DiffSynth-Studio
[BUG] Wan2.2 5B training demo failed in DiffSynth2.0
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
I'm encountering a training regression after upgrading to DIffSynth2.0. When running the official training script example/wanvideo/model_training/full/Wan2.2-TI2V-5B.sh, the resulting model generates severely distorted outputs, particularly in the first few frames of the generated video. See example output:
https://github.com/user-attachments/assets/771501fc-dd2b-4e76-8978-d2393690934f
However, when I downgrade back to the codebase to v1.19 (and use the corresponding training script from that release), training succeeds and produces expected results—no such artifacts appear. I have compared the corresponding codes but I have no idea about what makes the difference. I think it should a bug in 2.0. Can anyone help?
testing env: torch==2.5.1+cu12.4 torchvision==0.20.1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the regression with example/wanvideo/model_training/full/Wan2.2-TI2V-5B.sh under torch 2.5.1+cu12.4, then compare its behavior with the corresponding v1.19 training script. Investigate the code differences that affect the first generated video frames. Done means the DiffSynth2.0 training run no longer produces the reported distortions and matches the expected v1.19 output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100