NVIDIA / NVIDIA/TensorRT

FP16 failure of TensorRT 8.6.1.6 when running GroundingDINO on GPU GeForce RTX 3080 Ti

Open
#3,413 24 comments 0 reactions 2 assignees View on GitHub

@nvpohanh is already working on this.

Since Nov 4, 2023.

triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

I tried to convert GroundDINO onnx FP16 (op_set=17) to TensortRT FP16, the difference of the output is large.Then I tried to control input and output types of the INormalization layer, using the following code.
image
When converting model, it runs with the following error.It seems the problem with datatype,but i do not know how to fix this.
image

Environment

TensorRT Version:8.6.1.6

NVIDIA GPU:GeForce RTX 3080 Ti

NVIDIA Driver Version:515.43.04

CUDA Version:11.7

CUDNN Version:8.9.5.29

Operating System:Ubuntu20.04

Python Version (if applicable):python3.9

Tensorflow Version (if applicable):

PyTorch Version (if applicable):torch2.0.1

Have you tried the latest release?:I didn‘t find the TensorRT 9.1

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): ONNX model FP16 is fine.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.