modelscope / modelscope/DiffSynth-Studio

mu训练和推理对齐问题

Open
#1,248 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

def set_timesteps_flux2(num_inference_steps=100, denoising_strength=1.0, dynamic_shift_len=None):
        sigma_min = 1 / num_inference_steps
        sigma_max = 1.0
        num_train_timesteps = 1000
        sigma_start = sigma_min + (sigma_max - sigma_min) * denoising_strength  # 1.0
        sigmas = torch.linspace(sigma_start, sigma_min, num_inference_steps)  # [1.0000, 0.9990, ..., 0.0010]
        if dynamic_shift_len is None:  # TODO:看起来这是一个调整的参数mu,用于调整动态_shift_len的范围
            # If you ask me why I set mu=0.8,
            # I can only say that it yields better training results.
            mu = 0.8
        else:
            mu = FlowMatchScheduler.compute_empirical_mu(dynamic_shift_len, num_inference_steps)
        sigmas = math.exp(mu) / (math.exp(mu) + (1 / sigmas - 1))  # [1.0000, 0.9996, ..., 0.0022]
        timesteps = sigmas * num_train_timesteps
        return sigmas, timesteps

在跑flux2klein训练脚本时发现,训练时候dynamic_shift_len是None,但推理时为dynamic_shift_len=height // 16 * width // 16。这里为啥不需要对齐?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with set_timesteps_flux2 and trace the flux2klein training and inference call sites to compare how dynamic_shift_len is supplied. Determine whether the different values are intentional or cause inconsistent timesteps; done means the behavior is explained and any unintended mismatch is addressed with supporting coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.