invoke-ai / invoke-ai/InvokeAI

[enhancement]: Add support for INT8 ConvRot quantized models (*_int8_convrot.safetensors)

Open
#9,559 0 comments 0 reactions 1 assignee Claimed by @Pfannkuchensack View on GitHub
enhancement
Dominant language
Python
Stars
28.2k
Forks
3k
Avg merge
6d 5h
Merged PRs (30d)
19

Description

### Is there an existing issue for this?

- [x] I have searched the existing issues

### Contact Details

_No response_

### What should this feature add?

### Summary
I would like to request support for loading and inferencing **INT8 ConvRot** quantized models (e.g., `*_int8_convrot.safetensors`) in InvokeAI.

### Motivation & Background
Quantization using Hadamard / rotation matrices (`convrot`) has recently gained significant traction for large diffusion architectures (such as **Krea 2 Turbo/Raw**, **FLUX.1**, and other modern DiT pipelines).

Compared to standard FP8 and traditional INT8/GGUF approaches, `INT8 ConvRot`:
1. **Suppresses activation outliers** effectively, preserving near-lossless generation quality compared to BF16/FP16.
2. **Drastically reduces VRAM footprint** and enables high-resolution generations on consumer hardware (8GB - 12GB VRAM cards).
3. **Offers fast inference** with efficient linear dequantization kernels.

Currently, many newly released checkpoints (e.g., `krea2_turbo_int8_convrot.safetensors`) only work in ComfyUI (via native loader or `ComfyUI-INT8-Fast`). Having native dequantization support for `convrot` safetensors in InvokeAI would make these lightweight, high-performance weights accessible to InvokeAI users without requiring separate format conversions.

### Proposed Solution
- Add dynamic dequantization / loading support for tensors structured with rotated INT8 weights (`convrot`).
- Allow InvokeAI's Model Manager and unified loading pipeline to recognize and load `*_int8_convrot.safetensors` files without throwing structural mismatch or invalid layer errors.

### References & Existing Implementations
- **ComfyUI Integration:** Native support added in ComfyUI v0.27+ and community implementations like `ComfyUI-INT8-Fast`.
- **Target Models:** `Krea-2-Turbo`, `Krea-2-Raw`, `FLUX.1-dev/schnell` INT8 convrot checkpoints.

Thank you to the InvokeAI team for the continuous development and great work!

### Alternatives

_No response_

### Additional Content

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.