num_io_tensors get error of TensorRT 8.5 when running on GPU 4090
@zerollzeng is already working on this.
Since Apr 25, 2024.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
I have four input tensors [ "kpts0", "kpts1", "desc0", "desc1" ].
torch.onnx.export(
lightglue,
(kpts0, kpts1, desc0, desc1),
lightglue_path,
input_names=["kpts0", "kpts1", "desc0", "desc1"],
output_names=["matches0", "mscores0"],
opset_version=17,
dynamic_axes={
"kpts0": {1: "num_keypoints0"},
"kpts1": {1: "num_keypoints1"},
"desc0": {1: "num_keypoints0"},
"desc1": {1: "num_keypoints1"},
"matches0": {0: "num_matches0"},
"mscores0": {0: "num_matches0"},
},
)
I convert engine with the following command. onnx file
trtexec --onnx=superpoint_lightglue.onnx --saveEngine=superpoint_lightglue.engine
But when I use the Python API to obtain the IO Tensor, I only get desc0, desc1, matches0, mscores0.
import tensorrt as trt
logger = trt.Logger(trt.Logger.WARNING)
with open("superpoint_lightglue.engine", "rb") as f:
engine = trt.Runtime(logger).deserialize_cuda_engine(f.read())
tensor_names = [engine.get_tensor_name(i) for i in range(engine.num_io_tensors)]
print(tensor_names)
I get output as follow.
['desc0', 'desc1', 'matches0', 'mscores0']
Environment
TensorRT Version: v8.5.3 and v8.6.1
NVIDIA GPU: 4090
NVIDIA Driver Version: 535.129.03
CUDA Version: 11.8
CUDNN Version: 8.9.6
Operating System:
Python Version (if applicable): 3.11
Tensorflow Version (if applicable):
PyTorch Version (if applicable): 2.1.0
Baremetal or Container (if so, version):
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.