NVIDIA / NVIDIA/TensorRT

Conversion to TRT failure of TensorRT 8.6.1.6 when converting CO-DETR model on GPU RTX 4090

Open
#4,280 4 comments 0 reactions 1 assignee View on GitHub

@asfiyab-nvidia is already working on this.

Since Dec 18, 2024.

Module:Engine Build triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

I tried to convert model CO-DETR to TRT, but it fails with error below

[12/12/2024-02:17:38] [E] Error[10]: Could not find any implementation for node {ForeignNode[/0/Cast_3.../0/backbone/Reshape_3 + /0/backbone/Transpose_3]}.

Environment

TensorRT Version: 8.6.1.6

NVIDIA GPU: NVIDIA GeForce RTX 4090

NVIDIA Driver Version: 555.42.06

CUDA Version: 12.0

CUDNN Version:

Operating System: Ubuntu 22.04.3 LTS

Python Version (if applicable):

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version):

Relevant Files

Model link: https://drive.google.com/file/d/1voa7liji1OJxDQ8tphnbnm6EM-MexI_v/view?usp=drive_link

Steps To Reproduce

  • Env preparation: Build docker image following TensorRT-Docker-Image
  • Docker run and go to docker container
  • PyTorch to ONNX: follow DeepStream-Yolo
  • ONNX to TRT: trtexec --onnx=co_dino_5scale_swin_large_16e_o365tococo_h1280w1280.onnx --saveEngine=co_dino_5scale_swin_large_16e_o365tococo_h1280w1280.engine --explicitBatch --minShapes=input:1x3x1280x1280 --optShapes=input:2x3x1280x1280 --maxShapes=input:4x3x1280x1280 --fp16 --memPoolSize=workspace:10000 --tacticSources=-cublasLt,+cublas --sparsity=enable --verbose

Commands or scripts:

Have you tried the latest release?: Not yet.

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): If I follow steps described in DeepStream-Yolo, then the generated engine file works, but the speed is slow. Therefore, I would like to use trtexec.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.