modelscope / modelscope/DiffSynth-Studio

Question about Wan 2.2 High Noise Model Training

Open
#728 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

I used the Wan 2.2 full model training script in this repository and encountered the following problems during training:

When I train the Low Noise Model, the training results can be well adapted to the original High Noise Model, and the inference results are all normal.

But when I started training the High Noise Model, things changed. I found that the loss at this time is about dozens of times higher than the low training. At the same time, when I combined the trained High with the original Low, the inference result would be a lot of noise, as shown in the figure below:

Image

I'm not sure if anyone else has encountered similar problems. I guess training Lora should get the same conclusion as me. Is the training parameter configuration given by this repository the same as that used in the official training of Wan 2.2?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the repository's Wan 2.2 full model training script and compare the Low Noise and High Noise training parameter configurations. Reproduce the reported loss and noisy inference result, then check whether the repository settings match the official Wan 2.2 training configuration; done means identifying the configuration or implementation difference, if one exists.

Written by the indexing model from the issue text.

Assessment

Tech stack
machine-learning, python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.