FP16 failure of TensorRT 8.6.1.6 when running GroundingDINO on GPU GeForce RTX 3080 Ti
@nvpohanh is already working on this.
Since Nov 4, 2023.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
I tried to convert GroundDINO onnx FP16 (op_set=17) to TensortRT FP16, the difference of the output is large.Then I tried to control input and output types of the INormalization layer, using the following code.
When converting model, it runs with the following error.It seems the problem with datatype,but i do not know how to fix this.
Environment
TensorRT Version:8.6.1.6
NVIDIA GPU:GeForce RTX 3080 Ti
NVIDIA Driver Version:515.43.04
CUDA Version:11.7
CUDNN Version:8.9.5.29
Operating System:Ubuntu20.04
Python Version (if applicable):python3.9
Tensorflow Version (if applicable):
PyTorch Version (if applicable):torch2.0.1
Have you tried the latest release?:I didn‘t find the TensorRT 9.1
Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): ONNX model FP16 is fine.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.