Could not find any implementation for node Error while converting the instructor-large model from ONNX to TensorRT engine
@zerollzeng is already working on this.
Since Jul 25, 2023.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
We tried to convert the instructor-large onnx model to support TensorRT but we got errors:
[07/24/2023-03:37:25] [V] [TRT] --------------- Timing Runner: {ForeignNode[0.auto_model.shared.weight.../3/Div]} (Myelin[0x80000023])
[07/24/2023-03:38:05] [V] [TRT] Skipping tactic 0x0000000000000000 due to exception Internal bug. Please report with reproduction steps.
[07/24/2023-03:38:05] [V] [TRT] {ForeignNode[0.auto_model.shared.weight.../3/Div]} (Myelin[0x80000023]) profiling completed in 40.0429 seconds. Fastest Tactic: 0xd15ea5edd15ea5ed Time: inf
[07/24/2023-03:38:05] [E] Error[10]: Could not find any implementation for node {ForeignNode[0.auto_model.shared.weight.../3/Div]}.
[07/24/2023-03:38:05] [E] Error[10]: [optimizer.cpp::computeCosts::3869] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[0.auto_model.shared.weight.../3/Div]}.)
[07/24/2023-03:38:05] [E] Engine could not be created from network
[07/24/2023-03:38:05] [E] Building engine failed
[07/24/2023-03:38:05] [E] Failed to create engine from model or file.
[07/24/2023-03:38:05] [E] Engine set up failed
Environment
TensorRT Version: 8.6.1.6
NVIDIA GPU:T4
NVIDIA Driver Version: 535.54.03
CUDA Version: 12.1
CUDNN Version:8.9.2
Operating System:Unbuntu 22.04
Python Version (if applicable): Python 3.10
Baremetal or Container (if so, version):Container Version of TensoRT (23.06) docker image
Relevant Files
Model link: https://github.com/bingo-ctrl/onnx.git
Steps To Reproduce
Commands or scripts:
docker run -it --gpus all -v /path/to/instructor-large.onnx:/trt_optimize nvcr.io/nvidia/tensorrt:23.06-py3
trtexec --onnx=/trt_optimize/instructor-large.onnx --saveEngine=model.plan --explicitBatch --verbose
--minShapes='input_ids.1':1x32,'attention_mask.1':1x32,'context_masks.1':1
--optShapes='input_ids.1':1x64,'attention_mask.1':1x64,'context_masks.1':1
--maxShapes='input_ids.1':256x1024,'attention_mask.1':256x1024,'context_masks.1':256
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.