huggingface / huggingface/diffusers

Wan 2.1 / Wan2.2 Lightning & Sparse Distillation & Tiny VAE Acceleration Methods

Open
#12,146 18 comments 3 reactions 0 assignees View on GitHub
stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

Image

**Is your feature request related to a problem? Please describe.**
Some incredible acceleration methods have been released for both Wan2.1 and Wan2.2. These are currently appear to be unsupported with the official Diffusers pipelines.

- **[Wan2.2-Lightning](https://github.com/ModelTC/Wan2.2-Lightning)** enables 4-step inference for Wan2.2 via LoRAs
- **[LightX2V](https://github.com/ModelTC/lightx2v)** is related and also includes 4-step inference for Wan2.1 via LoRAs
- **[FastVideo](https://github.com/hao-ai-lab/FastVideo)** enables 3-step inference via distillation for Wan2.1 and Wan2.2 variants
- **[TAEHV](https://github.com/madebyollin/taehv?tab=readme-ov-file)** enables 4-6x increase in VAE decoding speed (2-3s to 0.5s)

**Describe the solution you'd like.**
1. The ability to load lightning LoRAs with Wan2.1 and Wan2.2 pipelines
2. The ability to load FastWan distilled models
3. The ability to load TAEHV as a Wan VAE alternative

**Describe alternatives you've considered.**

The [FastVideo](https://github.com/hao-ai-lab/FastVideo) and [LightX2V](https://github.com/ModelTC/lightx2v) official inference scripts.

TAEHV has a method for retro-fitting a [Diffusers pipeline](https://github.com/madebyollin/taehv/blob/main/examples/TAEW2.1_T2I_Demo.ipynb).

There are also various ComfyUI implementations.

**Additional context.**
Some of the LoRA acceleration might be supported already but is broken by recent changes to the LoRA loading ecosystem. Not certain. Also could use some guidance on editing the scheduler to allow for the time-step distillation.

I am currently investigating this space and figured I would open this issue to allow myself and others to post working examples and any relevant PRs. Thanks to all who do in advance.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.