NVIDIA / NVIDIA/TensorRT

using QDQonnx export engine file,fp16and int8，speed is not faster than using ONNX withouout QDQ to export fp16 engine

Open
#4,554 8 comments 0 reactions 1 assignee View on GitHub

@kevinch-nv is already working on this.

Since Aug 14, 2025.

Module:Performance triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Hello! I used mtq.quantize to quantize the RFDETR model and exported the QDQ ONNX. However, after converting it to an engine, the models exported with --int8 and --fp16, although smaller in size than the original fp16 engine file, have almost the same inference speed. Could you please suggest any solutions? Thank you.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.