onnx convert to TensorRT engin Error[1]: Unexpected exception bad allocation
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
Can't convert an onnx model to TensorRT engine under the following condition. It seems to be an error of memory allocation.
But when I change the GPU from 3070 to 2070 or use 3070 without --fp16, it works.
Environment
TensorRT Version: 8.2.0.6
NVIDIA GPU: RTX 3070
NVIDIA Driver Version: 469.49
CUDA Version: 11.4
CUDNN Version: 8.2.1
Operating System: windows 10
Python Version (if applicable):
Tensorflow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if so, version):
Relevant Files
Steps To Reproduce
trtexec.exe --onnx=model.onnx --saveEngine=model.engine --buildOnly --workspace=4096 --fp16
and I got the following error:
[11/09/2021-17:05:46] [I] Finish parsing network model
[11/09/2021-17:05:46] [I] [TRT] [MemUsageSnapshot] Builder begin: CPU 4097 MiB, GPU 2099 MiB
[11/09/2021-17:05:47] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +882, GPU +336, now: CPU 4994, GPU 2477 (MiB)
[11/09/2021-17:05:47] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +0, GPU +11, now: CPU 4994, GPU 2488 (MiB)
[11/09/2021-17:05:47] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[11/09/2021-17:12:13] [E] Error[1]: Unexpected exception bad allocation
[11/09/2021-17:12:13] [E] Error[2]: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::561] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
[11/09/2021-17:12:13] [E] Engine could not be created from network
[11/09/2021-17:12:13] [E] Building engine failed
[11/09/2021-17:12:13] [E] Failed to create engine from model.
[11/09/2021-17:12:13] [E] Engine set up failed
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source files or tests are named. Start by reproducing the exact trtexec command with model.onnx, then compare the RTX 3070 --fp16 failure with the working RTX 2070 or non-FP16 runs and inspect the reported allocation and engine-creation errors. Done means the supplied model builds successfully under the affected configuration or the failure is diagnosed with actionable evidence.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100