Project-MONAI / Project-MONAI/MONAI

Add support for model quantization.

Open
#8,205 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement Feature request
Dominant language
Python
Stars
8.7k
Forks
1.6k
Avg merge
5d 1h
Merged PRs (30d)
20

Description

Is your feature request related to a problem? Please describe.
In order to get a better inference performance, there are ways to quantize a deep learning model to a lower precision model like int8/int4 with an acceptable precsion decrease. Here are some examples:

  1. Pytorch Official
  2. NVIDIA library
  3. ONNX library

Medical images always cost plenty of inference time because of the 3D shape and large size. Since MONAI has already supported the onnx and trt export, it would be better to leverage the quantization feature supported by these formats and get a better latency for the medical image inference. What's more this will benefit the edge and network applications, both of which would be benefit from the low latency.

Describe the solution you'd like
APIs to convert, save, load and deploy quantization models.
Functions to perform the corresponding actions in python scripts.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing MONAI’s existing ONNX and TensorRT export support and the linked PyTorch, NVIDIA, and ONNX quantization documentation. Define Python APIs for converting, saving, loading, and deploying quantized models, with completion demonstrated by working quantization workflows for medical-image inference.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
api, machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.