modelscope / modelscope/DiffSynth-Studio
How were the VAE latent mean/std values obtained, and why are they used for latent scaling?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
Hi DiffSynth-Studio team,
Thank you for open-sourcing this project.
I noticed that the Qwen-Image/Wan pipeline applies fixed channel-wise normalization to the VAE latent:
latents = (latents - mean) / std
Could you clarify:
How were these mean and std values calculated?
Were they computed by encoding the training data, taking the posterior mean for each sample, and then calculating channel-wise statistics over all samples and spatial-temporal positions?
Why is this scaling necessary?
My guess is that the raw latent channels have different means and variances, so this normalization makes their scales more balanced, better matches the scale of Gaussian noise, and prevents high-variance channels from dominating the flow-matching loss.
It would be very helpful if you could confirm this or share the statistics-collection method. Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the Qwen-Image and Wan pipeline implementations where the channel-wise latent normalization is applied. Check the available VAE and training documentation or statistics-collection code to confirm how mean and standard deviation were obtained and why scaling is used; done means providing a clear, referenced explanation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100