TensorRT EP failed to set INT8 dynamic range.

Describe the issue

I followed the tutorial on: https://github.com/microsoft/onnxruntime-inference-examples/tree/main/quantization/nlp/bert/trt to generate an int8 model.
However, whenever I run inference, I get the following error:
2022-09-23 18:23:45.522261434 [E:onnxruntime:Default, tensorrt_execution_provider.h:58 log] [2022-09-23 18:23:45 ERROR] Setting dynamic range is only allowed when there are no Q/DQ layers in the Network. 2022-09-23 18:23:45.522306939 [E:onnxruntime:, sequential_executor.cc:368 Execute] Non-zero status code returned while running TRTKernel_graph_torch-jit-export_16130601706149353436_1 node. Name:'TensorrtExecutionProvider_TRTKernel_graph_torch-jit-export_16130601706149353436_1_0' Status Message: TensorRT EP failed to set INT8 dynamic range. EP Error: [ONNXRuntimeError] : 11 : EP_FAIL : Non-zero status code returned while running TRTKernel_graph_torch-jit-export_16130601706149353436_1 node. Name:'TensorrtExecutionProvider_TRTKernel_graph_torch-jit-export_16130601706149353436_1_0' Status Message: TensorRT EP failed to set INT8 dynamic range. using ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']

Running on TensorRT is crucial for our application, due to performance requirements.

Calibration files:
calibration.zip

I know these are two errors, but I would imagine the first one is causing the second one. I am not explicitly setting some dynamic range, so I am not sure what the issue is.

To reproduce

Tutorial on: https://github.com/microsoft/onnxruntime-inference-examples/tree/main/quantization/nlp/bert/trt

My scripts to generate the QDQ model follow the tutorial above. These are my own scripts (draft):
calibration scripts.zip

I cannot share the calibration data

Urgency

Blocking release, since a better int8 performance is required

Platform

Linux

OS Version

18.04

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.12.1

ONNX Runtime API

Python

Architecture

X86

Execution Provider

TensorRT

Execution Provider Library Version

Build cuda_11.3.r11.3/compiler.29920130_0