NVIDIA / NVIDIA/TensorRT

Could not find any implementation for node Error while converting the instructor-large model from ONNX to TensorRT engine

Open
#3,154 7 comments 0 reactions 1 assignee View on GitHub

@zerollzeng is already working on this.

Since Jul 25, 2023.

triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

We tried to convert the instructor-large onnx model to support TensorRT but we got errors:

[07/24/2023-03:37:25] [V] [TRT] --------------- Timing Runner: {ForeignNode[0.auto_model.shared.weight.../3/Div]} (Myelin[0x80000023])
[07/24/2023-03:38:05] [V] [TRT] Skipping tactic 0x0000000000000000 due to exception Internal bug. Please report with reproduction steps.
[07/24/2023-03:38:05] [V] [TRT] {ForeignNode[0.auto_model.shared.weight.../3/Div]} (Myelin[0x80000023]) profiling completed in 40.0429 seconds. Fastest Tactic: 0xd15ea5edd15ea5ed Time: inf
[07/24/2023-03:38:05] [E] Error[10]: Could not find any implementation for node {ForeignNode[0.auto_model.shared.weight.../3/Div]}.
[07/24/2023-03:38:05] [E] Error[10]: [optimizer.cpp::computeCosts::3869] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[0.auto_model.shared.weight.../3/Div]}.)
[07/24/2023-03:38:05] [E] Engine could not be created from network
[07/24/2023-03:38:05] [E] Building engine failed
[07/24/2023-03:38:05] [E] Failed to create engine from model or file.
[07/24/2023-03:38:05] [E] Engine set up failed

Environment

TensorRT Version: 8.6.1.6

NVIDIA GPU:T4

NVIDIA Driver Version: 535.54.03

CUDA Version: 12.1

CUDNN Version:8.9.2

Operating System:Unbuntu 22.04

Python Version (if applicable): Python 3.10

Baremetal or Container (if so, version):Container Version of TensoRT (23.06) docker image

Relevant Files

Model link: https://github.com/bingo-ctrl/onnx.git

Steps To Reproduce

Commands or scripts:

docker run -it --gpus all -v /path/to/instructor-large.onnx:/trt_optimize nvcr.io/nvidia/tensorrt:23.06-py3

trtexec --onnx=/trt_optimize/instructor-large.onnx --saveEngine=model.plan --explicitBatch --verbose
--minShapes='input_ids.1':1x32,'attention_mask.1':1x32,'context_masks.1':1
--optShapes='input_ids.1':1x64,'attention_mask.1':1x64,'context_masks.1':1
--maxShapes='input_ids.1':256x1024,'attention_mask.1':256x1024,'context_masks.1':256

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.