modelscope / modelscope/DiffSynth-Studio

LoRA fine-tuning on Wan2.2 high noise and low noise models has little to no effect — anyone else seeing this?

Open
#750 9 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

Hi, thanks for the amazing work and for open-sourcing Wan2.2!

I've been experimenting with LoRA fine-tuning on the Wan2.2 A14B model and noticed some unexpected behavior. I'm wondering if anyone else has encountered similar issues, or has insights to share.

I used the official training script to fine-tune both the low noise and high noise models for a clothing/makeup transformation task, with a timestep boundary set to 0.875. The training process completed without issues.

However, after fine-tuning, the results were very poor — I tested using both the official inference workflow and KJ’s inference workflow, but the model failed to learn motion or makeup transfer effectively.

In contrast, when I used the same dataset and task to fine-tune Wan2.1, the results were much better, and the LoRA successfully learned the desired transformations.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the report with the official Wan2.2 A14B training script, using the stated 0.875 timestep boundary for both noise models. Compare results through the official and KJ inference workflows against the Wan2.1 baseline; done means the cause of the ineffective LoRA learning is identified and the fix is validated on the clothing or makeup task.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.