huggingface / huggingface/diffusers
Wan 2.1 / Wan2.2 Lightning & Sparse Distillation & Tiny VAE Acceleration Methods
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
**Is your feature request related to a problem? Please describe.**
Some incredible acceleration methods have been released for both Wan2.1 and Wan2.2. These are currently appear to be unsupported with the official Diffusers pipelines.
- **[Wan2.2-Lightning](https://github.com/ModelTC/Wan2.2-Lightning)** enables 4-step inference for Wan2.2 via LoRAs
- **[LightX2V](https://github.com/ModelTC/lightx2v)** is related and also includes 4-step inference for Wan2.1 via LoRAs
- **[FastVideo](https://github.com/hao-ai-lab/FastVideo)** enables 3-step inference via distillation for Wan2.1 and Wan2.2 variants
- **[TAEHV](https://github.com/madebyollin/taehv?tab=readme-ov-file)** enables 4-6x increase in VAE decoding speed (2-3s to 0.5s)
**Describe the solution you'd like.**
1. The ability to load lightning LoRAs with Wan2.1 and Wan2.2 pipelines
2. The ability to load FastWan distilled models
3. The ability to load TAEHV as a Wan VAE alternative
**Describe alternatives you've considered.**
The [FastVideo](https://github.com/hao-ai-lab/FastVideo) and [LightX2V](https://github.com/ModelTC/lightx2v) official inference scripts.
TAEHV has a method for retro-fitting a [Diffusers pipeline](https://github.com/madebyollin/taehv/blob/main/examples/TAEW2.1_T2I_Demo.ipynb).
There are also various ComfyUI implementations.
**Additional context.**
Some of the LoRA acceleration might be supported already but is broken by recent changes to the LoRA loading ecosystem. Not certain. Also could use some guidance on editing the scheduler to allow for the time-step distillation.
I am currently investigating this space and figured I would open this issue to allow myself and others to post working examples and any relevant PRs. Thanks to all who do in advance.
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Begin by comparing the FastVideo and LightX2V official inference scripts with the Wan2.1 and Wan2.2 Diffusers pipelines. Review the TAEHV TAEW2.1_T2I_Demo.ipynb retrofit example and investigate the reported LoRA-loading and scheduler questions. Done means the pipelines can load the requested Lightning LoRAs, FastWan distilled models, and TAEHV VAE alternative with working examples.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python, pytorch
- Ambito
- machine-learning, performance
- Tipo di issue
- Funzionalità
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 25/100