NVIDIA / NVIDIA/TensorRT

convert onnx to trt, onnxparser fail, but trtexec success

Open
#2,084 12 comments 0 reactions 1 assignee View on GitHub

@zerollzeng is already working on this.

Since Jun 24, 2022.

triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

convert onnx to trt, use trtexec success, but onnxparser fail, error is : [06/23/2022-13:33:48] [E] [TRT] 10: [optimizer.cpp::computeCosts::1855] Error Code 10: Internal Error (Could not find any implementation for node TopK_531.)

the code use onnxparser is below, could you tell me where I did wrong:
`
if (!parser->parseFromFile(cfg.onnx.c_str(), static_cast(sample::gLogger.getReportableSeverity()))){
std::cerr << "Parsing File Failed" << std::endl;
return -1;
}
config->setMaxWorkspaceSize(cfg.workspace);
config -> setFlag(nvinfer1::BuilderFlag::kFP16);

auto profile = builder->createOptimizationProfile();
for(auto idx = cfg.minShapes.begin(); idx!=cfg.minShapes.end(); ++idx){
string input_name = (*idx).first;
if (cfg.maxShapes.count(input_name)==0 || cfg.optShapes.count(input_name)==0){
std::cerr<<"dyn shape need set minShapes, optShapes, maxShapes"<<std::endl;
return -1;
}
profile->setDimensions(input_name.c_str(), nvinfer1::OptProfileSelector::kMIN, nvinfer1::Dims4{cfg.minShapes[input_name][0], cfg.minShapes[input_name][1], cfg.minShapes[input_name][2], cfg.minShapes[input_name][3]});
profile->setDimensions(input_name.c_str(), nvinfer1::OptProfileSelector::kOPT, nvinfer1::Dims4{cfg.optShapes[input_name][0], cfg.optShapes[input_name][1], cfg.optShapes[input_name][2], cfg.optShapes[input_name][3]});
profile->setDimensions(input_name.c_str(), nvinfer1::OptProfileSelector::kMAX, nvinfer1::Dims4{cfg.maxShapes[input_name][0], cfg.maxShapes[input_name][1], cfg.maxShapes[input_name][2], cfg.maxShapes[input_name][3]});
}
config->addOptimizationProfile(profile);
auto CudaEngine = std::shared_ptrnvinfer1::ICudaEngine(builder->buildEngineWithConfig(*network, *config), InferDeleter());// got error here
`

Environment

TensorRT Version: 8.0.0.3
NVIDIA GPU: T4
NVIDIA Driver Version: 460.32.03
CUDA Version: 11.2
CUDNN Version: 8.0
Operating System: Ubuntu 20.04.3 LTS
Python Version (if applicable):
Tensorflow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if so, version):

Relevant Files

Steps To Reproduce

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.