openvinotoolkit / openvinotoolkit/nncf
[quantize_pt2e] Support batched statistics
@daniil-lyakhov is already working on this.
Since Jul 7, 2025.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 305
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 27
Description
🚀 Feature request
External quantizers could mark a multiply node as quantizable (see https://github.com/openvinotoolkit/nncf/pull/3558). NNCF is hardcoded the weight nodes via the weight_port_id parameter in metatypies, so the logic of the quantization is leaked to the metatypes.
The task is to figure out a way to correctly identify weight of a node without hardcoded parameter. For example, NNCF should identify which kind of multiply is present: activations x activations, weights x activations or activations x weights
The batched statistics should be enabled and tested for a custom quantizers (like x86)
Feature Use Case
Quantization with quantize_pt2e + custom quantizers with batch size > 1
Are you going to submit a PR?
- Yes I'd like to help by submitting a PR!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.