microsoft / microsoft/onnxruntime

Access violation when repeatedly creating/destroying inference session for TensorRT Execution Provider

Open
#24,529 10 comments 0 reactions 1 assignee Claimed by @chilo-ms View on GitHub
api:CSharp ep:TensorRT
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

## Describe the bug
Access violation error being thrown during the engine building phase when using TensorRT ExecutionProvider when repeatedly creating and deallocating the inference session.

## System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 x64
ONNX Runtime installed from (source or binary): binary
ONNX Runtime version: 1.19.2 AND 1.21.0
Python version: N/A (using C# wrapper)
Visual Studio version (if applicable): 2022
CUDA version:11.8.0 AND 12.8.1
cuDNN version: 8.9.7.29 AND 9.8.0.87
TensorRT version: 10.5.0.18 AND 10.8.0.43
GPU model and memory: RTX 3080

## To Reproduce

Unfortunately, we have not been able to reproduce the access violation isolated, but in our use case we are simply creating a model from onnx with TensorRT Execution Provider, run for a while, then destroy it. Run without. Then create a new. And then after some N repeating that for unknown reasons the access violation occurs. Only need to do it a handful of times. Always happens. Note model works fine usually and has for a few years. Basic CNN.

## Screenshots
![Image](https://github.com/user-attachments/assets/f20b58f0-1df3-4950-b3e7-4567993e3034)

## Code
https://github.com/microsoft/onnxruntime/blob/v1.19.2/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc#L2258

## Additional context

As can be seen from screenshot we have been trying hard to debug this, but we have hit a dead end, we do not understand why this occurs. Hence, this issue, in the hope we can get help finding the issue and remedying. Understandably, this is a bit hard given we have been unable to isolate code in question that actually reproduces despite trying this, we are unsure why this is not possible, since we do not do anything that different in our app itself.

Similar issue is [Access violation when using TensorRT ExecutionProvider on multiple GPU #7322](https://github.com/microsoft/onnxruntime/issues/7322) but no solution there for us it seems.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.