openvinotoolkit / openvinotoolkit/nncf
Add HQQ to weight compression algorithms for LLMs
@abhayuvi is already working on this.
Since Mar 13, 2026.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 305
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 27
Description
🚀 Feature request
HQQ is a popular data-free weight quantization algorithm for LLMs. It would be super cool to add it NNCF's weight compression algorithms. I would like to work on this myself. I understand I need to create my hqq.py file inside nncf/quantization/algorithms/weight_compression dir & I'm currently diving into the implementations of awq and gptq. Currently, I'm having trouble understanding the NNCFGraph object which needs to be passed to the apply method. Are there some docs on how to understand this Graph object? It would also be super helpful if you guys can point me to some code/docs that I can look into to understand the workflow better. Looking forward to contributing 🚀
Feature Use Case
HQQ is a fast and accurate model quantizer that skips the need for calibration data. It offers compression quality competitive with that of calibration-based methods. For instance, HQQ takes less than 5 minutes to process the colossal Llama-2-70B, that’s over 50x faster compared to the widely adopted GPTQ
Are you going to submit a PR?
- Yes I'd like to help by submitting a PR!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.