TensorRT 10.3 is 3+ times slower than p ytorch when running inference on Gpus A30 and 4090

Description

Under the same conditions, my model inference speed tensort is several times slower than pytorch

Environment

TensorRT Version: TensorRT.trtexec [TensorRT v100300]

NVIDIA GPU: A30 & 4090

NVIDIA Driver Version: 535.104.05

CUDA Version: release 12.4, V12.4.131

CUDNN Version: **

Operating System:

Python Version (if applicable):

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version):

Relevant Files

Model link:

https://drive.google.com/file/d/1V3wZFEyO6s3szE6tPhofa-bkY0Lqwu8M/view?usp=drive_link

Steps To Reproduce

./TensorRT-10.3.0.26/bin/trtexec --onnx=test_sim.onnx  --fp16 --shapes=phone:1x898x768,phone_lengths:1,pitch:1x898,pitchf:1x898,ds:1,rnd:1x192x898 --saveEngine=test.engine --builderOptimizationLevel=5

[08/26/2024-08:17:24] [I] GPU Compute Time: min = 817.994 ms, max = 820.003 ms, mean = 818.733 ms, median = 818.609 ms, percentile(90%) = 819.845 ms, percentile(95%) 
= 820.003 ms, percentile(99%) = 820.003 ms

pytorch uses the same input/output size, plus pre and post processing, and only needs 300ms

Start with the linked model and reproduce the TensorRT 10.3.0.26 trtexec command using the A30 or 4090 configuration reported here. Compare its 818 ms GPU compute time with the PyTorch benchmark under matching inputs, and verify the missing environment details. Done means isolating a reproducible performance cause or documenting the confirmed limitation with complete benchmark results.