Project-MONAI / Project-MONAI/MONAI
Add support for model quantization.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.7k
- Forks
- 1.6k
- Avg merge
- 5d 1h
- Merged PRs (30d)
- 20
Description
Is your feature request related to a problem? Please describe.
In order to get a better inference performance, there are ways to quantize a deep learning model to a lower precision model like int8/int4 with an acceptable precsion decrease. Here are some examples:
Medical images always cost plenty of inference time because of the 3D shape and large size. Since MONAI has already supported the onnx and trt export, it would be better to leverage the quantization feature supported by these formats and get a better latency for the medical image inference. What's more this will benefit the edge and network applications, both of which would be benefit from the low latency.
Describe the solution you'd like
APIs to convert, save, load and deploy quantization models.
Functions to perform the corresponding actions in python scripts.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing MONAI’s existing ONNX and TensorRT export support and the linked PyTorch, NVIDIA, and ONNX quantization documentation. Define Python APIs for converting, saving, loading, and deploying quantized models, with completion demonstrated by working quantization workflows for medical-image inference.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- api, machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100