❓ [Question] Model inference is much slower after updating to TensorRT 9.3

❓ Question

I have a VIT model for object detection. The model inference speed in the tensort 8.5 environment is 190ms per frame. However when I updated to TensorRT 9.3, Inference slowed down to 250ms per frame.

I acquired the C++ dynamic library by compiling the latest Torch-TensorRT source code.

What might be causing this issue?

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

• Libtorch Version (e.g., 1.0): 2.2.1
• CPU Architecture:
• OS (e.g., Linux): ubuntu22.04
• How you installed PyTorch (conda, pip, libtorch, source):
• Build command you used (if compiling from source):
• Are you using local sources or building from archives: Yes
• Python version:
• CUDA version: 12.2
• GPU models and configuration:
• Any other relevant information:

No source file or test is identified. Start by reviewing the reported Torch-TensorRT build context and comparing the VIT inference measurements between TensorRT 8.5 and 9.3, filling in the missing GPU, build, installation, and CPU details. Done means the slowdown has a reproducible configuration and an identified cause.