huggingface / huggingface/diffusers

Wan i2i with latent upscale

Open
#12,173 1 comment 1 reaction 0 assignees View on GitHub
stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

### Model/Pipeline/Scheduler description

I made a wan i2i pipeline that can also somewhat handle upscaling (not really a true upscaler since it's very creative). It only uses the low noise transformer and can generate good results in 20 steps. It's useful to transform low quality pictures and to change the overall vibe of the composition leveraging the low noise transformer in Wan to add details.

A few examples:

* small upscale, simple mood change: https://1nf.sh/tasks/30dt0n0npdx5fa434cpryfs7je
* 5x upscale from low res (200px) : https://1nf.sh/tasks/19t5ah8bf27y8fqwsdyh058hz4

### Open source status

- [x] The model implementation is available.
- [x] The model weights are available (Only relevant if addition is not a scheduler).

### Provide useful links for the implementation

Not really sure there is appetite to have it in diffusers but in case code is here if you want to play with it https://gist.github.com/luke14free/33365d5fc4ffc1b83e18a5d0c4094b1a - its not perfect especially at low noise strengths & high scales & low cfg (it's not a good upscaler unless you allow it to be creative). If you happen to try it, feedback on the implementation is also appreciated, especially around those low strengths.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.