Please add support for M.I.T SVDQuant. This will ensure that the Int4 model quality matches that of the FP8 model.
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 155
Description
### Feature Idea
demo: https://svdquant.mit.edu/flux1-schnell/
This is a new W4A4 method for Int4 models, allowing you to convert models to this Int4 version. It can achieve the same quality as the FP8 method in the Flux model .
FP8 Flux 20steps in 1m58s,
**SVDQ-Int4 20steps in 30s. Quality better than Q4.GGUF**
Requires less than 6GB, It has tested on my 3060 laptop.
https://github.com/mit-han-lab/nunchaku
In Windows, you can install the WHL file using this plugin to make it work, but there may be issues with other plugins. ComfyUI might offer a solution.

### Existing Solutions
_No response_
### Other
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.