oneapi-src / oneapi-src/oneAPI-samples
How to Perform Static Quantization Directly on an ONNX Model Using Intel® Neural Compressor?
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.2k
- Forks
- 745
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
I'm using Intel® Neural Compressor (INC) to perform static quantization on my custom PyTorch model. I followed this script which demonstrates how to apply static quantization using INC on a PyTorch model.
My goal is to obtain the final quantized model in ONNX format. However, after quantization, saving the q_model results in a .pt file (PyTorch format). I also found that exporting quantized PyTorch models to ONNX is problematic due to limited support and compatibility issues, especially with static quantization.
My Question:
Is there a way to perform static quantization directly on an ONNX model using Intel® Neural Compressor to produce a quantized ONNX model as the output?
Alternatively, is there a specific method to export the statically quantized PyTorch model to ONNX format while addressing the compatibility issues?
Any guidance or examples on how to achieve this would be greatly appreciated.
Thank you!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked quantize_with_inc.ipynb sample and inspect how it saves the quantized PyTorch model. Verify whether the requested ONNX input/output path is supported and whether an export example belongs in this sample. Done means a clear, reproducible answer or example covering direct ONNX quantization and the alternative export path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100