modelscope / modelscope/DiffSynth-Studio

Support Perceptual Flow Matching (PFM) for Few-Step Generation and Distillation

Open
#1,518 5 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

Hi! We recently published our work, Perceptual Flow Matching for Few-Step Generative Modeling: https://arxiv.org/abs/2607.03524.

In this work, we show that simply replacing the standard MSE loss with a perceptual loss during flow matching training can produce a strong few-step generator with only a few hundred training steps. We have validated the method on SD3, Qwen-Image-Edit, and Wan. The Qwen-Image-Edit and Wan experiments were both implemented based on DiffSynth-Studio.

I was wondering whether you would be interested in integrating PFM into DiffSynth-Studio. The method is highly compatible with the current SFT training pipeline, as it only requires replacing the training loss while leaving the rest of the framework unchanged.

If this sounds interesting, I'd be happy to help integrate our existing implementation and submit a PR for the project.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start by locating the current SFT training pipeline and its training-loss implementation, then compare it with the authors' existing PFM implementation. Done means PFM can replace the standard loss in the pipeline and support the validated SD3, Qwen-Image-Edit, and Wan use cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.