"This version of TensorRT does not support dynamic axes." failure of TensorRT 8.5.2 and 8.6.1 when running groundingdino.onnx on GPU Tesla V100 and Tesla T4

Description

I tried to convert grouding.onnx to tensorrt on GPU, but it fails with the error below
torch2onnx commend:

            caption =  "the running dog ." #". ".join(input_text)
            input_ids =  model.tokenizer([caption], return_tensors="pt")["input_ids"]
            position_ids = torch.tensor([[0, 0, 1, 2, 3, 0]])
            token_type_ids = torch.tensor([[0, 0, 0, 0, 0, 0]])
            attention_mask = torch.tensor([[True, True, True, True, True, True]])
            text_token_mask = torch.tensor([[[ True, False, False, False, False, False],
                [False,  True,  True,  True,  True, False],
                [False,  True,  True,  True,  True, False],
                [False,  True,  True,  True,  True, False],
                [False,  True,  True,  True,  True, False],
                [False, False, False, False, False,  True]]])
            
            img = torch.randn(1, 3, 512, 512)
            # img = image[None]

            dynamic_axes={
            "input_ids": {0: "batch_size", 1: "seq_len"},
            "attention_mask": {0: "batch_size", 1: "seq_len"},
            "position_ids": {0: "batch_size", 1: "seq_len"},
            "token_type_ids": {0: "batch_size", 1: "seq_len"},
            "text_token_mask": {0: "batch_size", 1: "seq_len", 2: "seq_len"},       
            "img": {0: "batch_size", 2: "height", 3: "width"},
            "logits": {0: "batch_size"},
            "boxes": {0: "batch_size"}
            }
            # output_model = model(img, input_ids, attention_mask, position_ids, token_type_ids, text_token_mask)
            # print(f'output_model:{output_model}')
            # outputs = output_model
            onnx_path = "/GroundingDINO/weights/gd_token_dynamic_sigmoid_512_fold.onnx"
            torch.onnx.export(
                            model,
                            f=onnx_path,
                            args=(img, input_ids, attention_mask, position_ids, token_type_ids, text_token_mask), #, zeros, ones),
                            input_names=["img" , "input_ids", "attention_mask", "position_ids", "token_type_ids", "text_token_mask"],
                            output_names=["logits", "boxes"],
                            do_constant_folding=True, 
                            dynamic_axes=dynamic_axes,
                            opset_version=16)

( error log).

[12/15/2023-10:39:35] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/15/2023-10:39:36] [W] [TRT] onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/15/2023-10:39:37] [E] [TRT] ModelImporter.cpp:726: While parsing node number 3432 [Slice -> "onnx::Slice_3626"]:
[12/15/2023-10:39:37] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[12/15/2023-10:39:37] [E] [TRT] ModelImporter.cpp:728: input: "onnx::Slice_3616"
input: "onnx::Slice_22788"
input: "onnx::Slice_3622"
input: "onnx::Slice_22789"
input: "onnx::Slice_3625"
output: "onnx::Slice_3626"
name: "Slice_3432"
op_type: "Slice"

[12/15/2023-10:39:37] [E] [TRT] ModelImporter.cpp:729: --- End node ---
[12/15/2023-10:39:37] [E] [TRT] ModelImporter.cpp:732: ERROR: builtin_op_importers.cpp:4531 In function importSlice:
[8] Assertion failed: (axes.allValuesKnown()) && "This version of TensorRT does not support dynamic axes."
[12/15/2023-10:39:37] [E] Failed to parse onnx file
[12/15/2023-10:39:37] [I] Finish parsing network model
[12/15/2023-10:39:37] [E] Parsing model failed
[12/15/2023-10:39:37] [E] Failed to create engine from model or file.
[12/15/2023-10:39:37] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8503] # trtexec --onnx=grounded_1207.onnx --saveEngine=grounded_1207.plan

Environment

TensorRT Version:8.5.3

NVIDIA GPU:Tesla T4

NVIDIA Driver Version:470.103.01

CUDA Version:11.4

CUDNN Version:

Environment2（I tried two Environment2 but got the same error）

TensorRT Version:8.6.1

NVIDIA GPU:Tesla V100

NVIDIA Driver Version:470.103.01

CUDA Version:11.4

CUDNN Version:

Operating System:

Python Version (if applicable): 3.8

PyTorch Version (if applicable):1.12.0

Relevant Files

Model link:
https://github.com/IDEA-Research/GroundingDINO/issues/46

Steps To Reproduce

Commands or scripts:

Have you tried the latest release?:
Yes, I tried tensorrt 8.6.1. but got the same error

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):
Yes, I run the onnx model with ONNXRuntime and got same results with ckpt'results