NVIDIA / NVIDIA/TensorRT

num_io_tensors get error of TensorRT 8.5 when running on GPU 4090

Open
#3,803 7 comments 0 reactions 1 assignee View on GitHub

@zerollzeng is already working on this.

Since Apr 25, 2024.

triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

I have four input tensors [ "kpts0", "kpts1", "desc0", "desc1" ].

torch.onnx.export(
            lightglue,
            (kpts0, kpts1, desc0, desc1),
            lightglue_path,
            input_names=["kpts0", "kpts1", "desc0", "desc1"],
            output_names=["matches0", "mscores0"],
            opset_version=17,
            dynamic_axes={
                "kpts0": {1: "num_keypoints0"},
                "kpts1": {1: "num_keypoints1"},
                "desc0": {1: "num_keypoints0"},
                "desc1": {1: "num_keypoints1"},
                "matches0": {0: "num_matches0"},
                "mscores0": {0: "num_matches0"},
            },
        )

I convert engine with the following command. onnx file

trtexec --onnx=superpoint_lightglue.onnx --saveEngine=superpoint_lightglue.engine

But when I use the Python API to obtain the IO Tensor, I only get desc0, desc1, matches0, mscores0.

import tensorrt as trt

logger = trt.Logger(trt.Logger.WARNING)

with open("superpoint_lightglue.engine", "rb") as f:
        engine = trt.Runtime(logger).deserialize_cuda_engine(f.read())
        tensor_names = [engine.get_tensor_name(i) for i in range(engine.num_io_tensors)]
        print(tensor_names)

I get output as follow.

['desc0', 'desc1', 'matches0', 'mscores0']

Environment

TensorRT Version: v8.5.3 and v8.6.1

NVIDIA GPU: 4090

NVIDIA Driver Version: 535.129.03

CUDA Version: 11.8

CUDNN Version: 8.9.6

Operating System:

Python Version (if applicable): 3.11

Tensorflow Version (if applicable):

PyTorch Version (if applicable): 2.1.0

Baremetal or Container (if so, version):

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.