invoke-ai / invoke-ai/InvokeAI
[enhancement]: Add support for INT8 ConvRot quantized models (*_int8_convrot.safetensors)
- Dominant language
- Python
- Stars
- 28.2k
- Forks
- 3k
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 19
Description
### Is there an existing issue for this?
- [x] I have searched the existing issues
### Contact Details
_No response_
### What should this feature add?
### Summary
I would like to request support for loading and inferencing **INT8 ConvRot** quantized models (e.g., `*_int8_convrot.safetensors`) in InvokeAI.
### Motivation & Background
Quantization using Hadamard / rotation matrices (`convrot`) has recently gained significant traction for large diffusion architectures (such as **Krea 2 Turbo/Raw**, **FLUX.1**, and other modern DiT pipelines).
Compared to standard FP8 and traditional INT8/GGUF approaches, `INT8 ConvRot`:
1. **Suppresses activation outliers** effectively, preserving near-lossless generation quality compared to BF16/FP16.
2. **Drastically reduces VRAM footprint** and enables high-resolution generations on consumer hardware (8GB - 12GB VRAM cards).
3. **Offers fast inference** with efficient linear dequantization kernels.
Currently, many newly released checkpoints (e.g., `krea2_turbo_int8_convrot.safetensors`) only work in ComfyUI (via native loader or `ComfyUI-INT8-Fast`). Having native dequantization support for `convrot` safetensors in InvokeAI would make these lightweight, high-performance weights accessible to InvokeAI users without requiring separate format conversions.
### Proposed Solution
- Add dynamic dequantization / loading support for tensors structured with rotated INT8 weights (`convrot`).
- Allow InvokeAI's Model Manager and unified loading pipeline to recognize and load `*_int8_convrot.safetensors` files without throwing structural mismatch or invalid layer errors.
### References & Existing Implementations
- **ComfyUI Integration:** Native support added in ComfyUI v0.27+ and community implementations like `ComfyUI-INT8-Fast`.
- **Target Models:** `Krea-2-Turbo`, `Krea-2-Raw`, `FLUX.1-dev/schnell` INT8 convrot checkpoints.
Thank you to the InvokeAI team for the continuous development and great work!
### Alternatives
_No response_
### Additional Content
_No response_
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.