Inference failure of TensorRT 10.0.x.x when running my internal model on GPU(T4, A100)

Description

After updating to TensorRT 10.0.1.6, we expected the previously reported issue to be resolved. Unfortunately, not only does the issue persist, but the model’s outputs have deteriorated even further. Specifically, all output values are now nan, making it impossible to use our models. This issue affects both fp16 and fp32 precision settings, rendering the model completely non-functional.

https://github.com/NVIDIA/TensorRT/issues/3292

Environment

TensorRT Version: All version of 10.0.x.x. NGC Container 24.05~24.07.

NVIDIA GPU: T4, A100

NVIDIA Driver Version: 550.90.07

CUDA Version: 12.4

CUDNN Version: x

Operating System:

Container (if so, version): NGC Container from 23.03 and 24.07.
https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tensorrt

Operating System:

Python Version (if applicable):

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version):

Relevant Files

Model link:

Steps To Reproduce

Commands or scripts:

Have you tried the latest release?:

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):

fail on polygraphy

related to https://github.com/NVIDIA/TensorRT/issues/3292

No model link, script, command, traceback, or named test is provided. Start by comparing the internal model with issue #3292 and collecting an exact Polygraphy reproduction in the listed TensorRT, CUDA, driver, GPU, and container environments. Done requires a minimal reproducible case that demonstrates the NaN outputs and identifies the failing inference path.