invoke-ai / invoke-ai/InvokeAI
[enhancement]: Optional acceleration for non-distilled models
- Dominant language
- Python
- Stars
- 28.2k
- Forks
- 3k
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 19
Description
### Is there an existing issue for this?
- [x] I have searched the existing issues
### Contact Details
_No response_
### What should this feature add?
It may be helpful to add support for easy access to distillation LoRAs on non-distilled architectures, e.g. for Strix Halo users and those with lower-end cards that have sufficient memory but slower rendering speeds. I reckon that this would be best done in stages:
1. Testing support for acceleration LoRAs (noting https://github.com/invoke-ai/InvokeAI/issues/8415)
2. Adding relevant LoRAs to the Starter Models list
3. Adding a checkbox to the linear UI that can be activated by users when desired
4. Guarding against incompatible CFG settings when acceleration is activated
Here is a list of likely LoRAs for inclusion:
| Architecture | Steps | Huggingface Repository |
|--------------------|----------|------------------------------------|
| Stable Diffusion 1.5 | 8 (CFG preserved)| [ByteDance/Hyper-SD](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SD15-8steps-CFG-lora.safetensors) |
| Stable Diffusion XL | 8 (CFG preserved) | [ByteDance/Hyper-SD](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-8steps-CFG-lora.safetensors) |
| Stable Diffusion 3.5 Medium | 4 (requires CFG=1) | [tensorart/stable-diffusion-3.5-medium-turbo](https://huggingface.co/tensorart/stable-diffusion-3.5-medium-turbo/blob/main/lora_sd3.5m_4steps.safetensors)
| Flux.1 Dev | 4* | [Lingyuzhou/Hyper_Flux.1_Dev_4_step_Lora](https://huggingface.co/Lingyuzhou/Hyper_Flux.1_Dev_4_step_Lora/blob/main/Hyper-Flux.1-Dev%204-step-Lora.safetensors) |
| Anima | 4 (requires CFG=1) | [sorryhyun/anima-turbo-4step](https://huggingface.co/sorryhyun/anima-turbo-4step/blob/main/anima_turbo_4step_v2_longer.safetensors) |
| Krea 2 Turbo | 4* | [lvladikov/Krea2-Turbo-Distill-4step-LoRA](https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA/blob/main/krea2_turbo_4step_rank_64_lora.safetensors) |
\* Base model already distilled
### Alternatives
The CFG-preserved LoRAs listed usually have lower-step alternatives that require the CFG to be set to 1. I have opted here for the more conservative options.
### Additional Content
- Fooocus previously [bundled](https://github.com/mashb1t/Fooocus/blob/21a44c214a574536216e03209ee034f35c6527c9/update_log.md#240) the 4-step LoRA with its SDXL implementation, making it available via a preset
- Stable Diffusion 3.5 Large has a [4-step Turbo model](https://huggingface.co/stabilityai/stable-diffusion-3.5-large-turbo) rather than an acceleration LoRA (but note #9399)
- If this FR is implemented, it may be possible to support CPU-only generation on select architectures using 1-step LoRAs, as demonstrated in [this](https://github.com/rupeshs/fastsdcpu) existing Diffusers implementation
- [SD1.5](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SD15-1step-lora.safetensors)
- [SDXL](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-1step-lora.safetensors)
- [Flux.2 Klein 4B](https://huggingface.co/epfl-vita/flux2-klein-1step-rdm)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.