modelscope / modelscope/DiffSynth-Studio

How were the VAE latent mean/std values obtained, and why are they used for latent scaling?

Open
#1,536 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

Hi DiffSynth-Studio team,

Thank you for open-sourcing this project.

I noticed that the Qwen-Image/Wan pipeline applies fixed channel-wise normalization to the VAE latent:

latents = (latents - mean) / std

Could you clarify:

How were these mean and std values calculated?
Were they computed by encoding the training data, taking the posterior mean for each sample, and then calculating channel-wise statistics over all samples and spatial-temporal positions?
Why is this scaling necessary?

My guess is that the raw latent channels have different means and variances, so this normalization makes their scales more balanced, better matches the scale of Gaussian noise, and prevents high-variance channels from dominating the flow-matching loss.

It would be very helpful if you could confirm this or share the statistics-collection method. Thanks!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the Qwen-Image and Wan pipeline implementations where the channel-wise latent normalization is applied. Check the available VAE and training documentation or statistics-collection code to confirm how mean and standard deviation were obtained and why scaling is used; done means providing a clear, referenced explanation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.