fp16 engine works fine with trtexec but encountering nan with python API

Description

I try to create a tensorrt engine from an onnx model

trtexec --onnx=model.onnx --saveEngine=engine.trt --fp16

When i use trtexec for inference it works fine

trtexec --loadEngine=engine.trt  --fp16  --exportOutput=f
p16e.json

I can see that the output is just fine
fp16e.json.

But when I try to run this engine file with python API the output is nan.

Environment

TensorRT Version: 8.6

NVIDIA GPU: RTX 3070

NVIDIA Driver Version: 535.154.05

CUDA Version: 12.2

CUDNN Version: 8.9

Operating System:

Python Version (if applicable): 3.10

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version):

Relevant Files

Model link:

Steps To Reproduce

Commands or scripts:

Here is the python script I am using tensorrt_test_fps.py.txt

Have you tried the latest release?: yes

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):
Yes it runs fine when using CUDAExecutionProvider with ONNXRuntime but with TensorrtExecutionProvider I get the same issue.