NVIDIA / NVIDIA/TensorRT

Question: Disable Optimizations for TensorRT

Open
#4,075 5 comments 0 reactions 1 assignee View on GitHub

@zhenhuaw-me is already working on this.

Since Feb 11, 2025.

Investigating Module:Engine Build
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Question

Because there are so many optimizations that TRT performs, sometimes it is very hard to isolate the issue if we see regression in model accuracy. I know we have the builder_optimization_level flag, but it seems to only control which kernel is used when executing the model.

I wonder if there is more fine-grained control? For example, I want to prevent fusions, or prevent removing dead code.

To give more context: In my specific use case, I am interested in isolating whether resolving Q/DQ nodes can causes regression in model. What I am interested to achieve is to only enable Q/DQ resolution and disable all other optimizations. Is this achievable?

Thanks in advance

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.