huggingface / huggingface/diffusers
Wan 2.1 / Wan2.2 Lightning & Sparse Distillation & Tiny VAE Acceleration Methods
- Dominant language
- Python
- Stars
- 34.5k
- Forks
- 7.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 91
Description
**Is your feature request related to a problem? Please describe.**
Some incredible acceleration methods have been released for both Wan2.1 and Wan2.2. These are currently appear to be unsupported with the official Diffusers pipelines.
- **[Wan2.2-Lightning](https://github.com/ModelTC/Wan2.2-Lightning)** enables 4-step inference for Wan2.2 via LoRAs
- **[LightX2V](https://github.com/ModelTC/lightx2v)** is related and also includes 4-step inference for Wan2.1 via LoRAs
- **[FastVideo](https://github.com/hao-ai-lab/FastVideo)** enables 3-step inference via distillation for Wan2.1 and Wan2.2 variants
- **[TAEHV](https://github.com/madebyollin/taehv?tab=readme-ov-file)** enables 4-6x increase in VAE decoding speed (2-3s to 0.5s)
**Describe the solution you'd like.**
1. The ability to load lightning LoRAs with Wan2.1 and Wan2.2 pipelines
2. The ability to load FastWan distilled models
3. The ability to load TAEHV as a Wan VAE alternative
**Describe alternatives you've considered.**
The [FastVideo](https://github.com/hao-ai-lab/FastVideo) and [LightX2V](https://github.com/ModelTC/lightx2v) official inference scripts.
TAEHV has a method for retro-fitting a [Diffusers pipeline](https://github.com/madebyollin/taehv/blob/main/examples/TAEW2.1_T2I_Demo.ipynb).
There are also various ComfyUI implementations.
**Additional context.**
Some of the LoRA acceleration might be supported already but is broken by recent changes to the LoRA loading ecosystem. Not certain. Also could use some guidance on editing the scheduler to allow for the time-step distillation.
I am currently investigating this space and figured I would open this issue to allow myself and others to post working examples and any relevant PRs. Thanks to all who do in advance.
Contributor guide
Assessment
This issue has not been assessed yet.