Bringing Back Effective Quantization: Using ModelOPT for YOLO and Similar Architectures

I have developed a project YOLOv9-QAT to perform quantization using QAT for models based on the YOLO architecture using discontinued nvidia pytorch-quantization library.

However, I faced several challenges while using modelopt and noticed that some functions have been changed. Due to the model's complexity and the need for greater control over the insertion of QDQ nodes, we had to configure it in a rule-based manner, ensuring that the setup is optimal under TensorRT. This involved making sure that all nodes are running in INT8.

I have seen many recommendations to use ONNX PTQ Toolkit to quantize ONNX models, but the results have been unsatisfactory.

It would be interesting to revive this repository: https://github.com/NVIDIA-AI-IOT/yolo_deepstream/tree/main/yolov7_qat as an example for performing quantization in FP8, INT8, or INT4 on models with similar architectures, such as ResNet, among others. This would help the community implement quantization for detection and segmentation models that use similar architectures.

Another point is that exporting the QAT model from PyTorch to ONNX offers flexibility for exporting using TensorRT plugins, such as EfficientNMS, among others. However, if I try to quantize ONNX models exported with plugins using the ONNX PTQ Toolkit, it results in an error during quantization. This forces me to perform quantization directly in PyTorch and then export to ONNX.

My main goal is to get a direction to achieve the same or better results using modelopt as we did with the pytorch-quantization library.

Results:

Evaluation Results

Activation SiLU

Activation ReLU