huggingface / huggingface/diffusers

[Scheduler] Add SNR shift following SD3, would the rest of the code need to be modified?

Open
#9,392 7 comments 0 reactions 0 assignees View on GitHub
stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

**What API design would you like to have changed or added to the library? Why?**

With the increasing resolution of image or video generation, we need to introduce more noise at smaller T, such as SNR shift following SD3. I have observed that CogVideoX's schedule has already implemented [this](https://github.com/huggingface/diffusers/blob/main/src/diffusers/schedulers/scheduling_ddim_cogvideox.py#L214). If I add this line to the DDPM schedule, would the rest of the code (e.g., noise addition, sampling, etc.) need to be modified? I assume it wouldn't, but I seek a precise response.

**What use case would this enable or better enable? Can you give us a code example?**

```
class DDPMScheduler(SchedulerMixin, ConfigMixin):
def __init__(snr_shift_scale, **kwarg)
# predefine beta and alpha
self.alphas_cumprod = self.alphas_cumprod / (snr_shift_scale + (1 - snr_shift_scale) * self.alphas_cumprod)
# other code
# Other functions are the same as before
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.