FP16 Accuracy failure of TensorRT 8.6.3 when running trtexec built engine on GPU RTX4090

Description

We are trying to recreate the results from: https://arxiv.org/abs/2402.05008.

Using the same .onnx file to compile engines provided by the authors, we find that using fp16 model has 0% accuracy while the fp32 model has accuracy as expected. This was previously possible for us, but now compiling the model in fp16 is not working.

The measurements are mIoU scores for different sized objects.

L2 - FP16
{"all": 0.0, "large": 0.0, "medium": 0.0, "small": 0.0}

L2 - FP32
{"all": 79.12385607181146, "large": 83.05853600575689, "medium": 81.50597370444349, "small": 74.8830670481846}

Environment

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.14              Driver Version: 550.54.14      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        Off |   00000000:01:00.0 Off |                  Off |
| 40%   32C    P8              6W /  450W |      11MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

Baremetal or Container (if so, version): tensort-24.02-py

Relevant Files

engine inspection:
l2__fp16_inspect.txt
l2_fp32_inspect.txt

build log:
l2_fp32.log
l2_fp16.log

onnx link: https://drive.google.com/drive/folders/1Yt8xDfdkmL6W-IO-KhUhR_J-_2ion-v5?usp=sharing