NVIDIA / NVIDIA/TensorRT

onnx convert to TensorRT engin Error[1]: Unexpected exception bad allocation

Open
#1,603 8 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

Can't convert an onnx model to TensorRT engine under the following condition. It seems to be an error of memory allocation.
But when I change the GPU from 3070 to 2070 or use 3070 without --fp16, it works.

Environment

TensorRT Version: 8.2.0.6
NVIDIA GPU: RTX 3070
NVIDIA Driver Version: 469.49
CUDA Version: 11.4
CUDNN Version: 8.2.1
Operating System: windows 10
Python Version (if applicable):
Tensorflow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if so, version):

Relevant Files

model.onnx

Steps To Reproduce

trtexec.exe --onnx=model.onnx --saveEngine=model.engine --buildOnly --workspace=4096 --fp16
and I got the following error:

[11/09/2021-17:05:46] [I] Finish parsing network model
[11/09/2021-17:05:46] [I] [TRT] [MemUsageSnapshot] Builder begin: CPU 4097 MiB, GPU 2099 MiB
[11/09/2021-17:05:47] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +882, GPU +336, now: CPU 4994, GPU 2477 (MiB)
[11/09/2021-17:05:47] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +0, GPU +11, now: CPU 4994, GPU 2488 (MiB)
[11/09/2021-17:05:47] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[11/09/2021-17:12:13] [E] Error[1]: Unexpected exception bad allocation
[11/09/2021-17:12:13] [E] Error[2]: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::561] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
[11/09/2021-17:12:13] [E] Engine could not be created from network
[11/09/2021-17:12:13] [E] Building engine failed
[11/09/2021-17:12:13] [E] Failed to create engine from model.
[11/09/2021-17:12:13] [E] Engine set up failed

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source files or tests are named. Start by reproducing the exact trtexec command with model.onnx, then compare the RTX 3070 --fp16 failure with the working RTX 2070 or non-FP16 runs and inspect the reported allocation and engine-creation errors. Done means the supplied model builds successfully under the affected configuration or the failure is diagnosed with actionable evidence.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.