microsoft / microsoft/onnxruntime

C++ inference with GPU (CUDA)

Open
#13,934 13 comments 0 reactions 0 assignees View on GitHub
documentation ep:CUDA platform:windows
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

Small program

int main(int argc, char *argv[])
{
Ort::Env env;
Ort::SessionOptions session_options;
OrtCUDAProviderOptions cuda_options;
cuda_options.device_id = 0;
session_options.AppendExecutionProvider_CUDA(cuda_options);
Ort::Session session = Ort::Session(env, L"rim.onnx", session_options);
std::cout << "End reached" << std::endl;
}

running this program, we have no output (also not any error or something like that). When taking out the AppendExecutionProvider_CUDA call it works fine.
Any idea how to debug this kind of problem?
I'm using VS 2019 & onnxruntime 1.13.1 with cuda 11.4

### To reproduce

int main(int argc, char *argv[])
{
Ort::Env env;
Ort::SessionOptions session_options;
OrtCUDAProviderOptions cuda_options;
cuda_options.device_id = 0;
session_options.AppendExecutionProvider_CUDA(cuda_options);
Ort::Session session = Ort::Session(env, L"rim.onnx", session_options);
std::cout << "End reached" << std::endl;
}

### Urgency

_No response_

### Platform

Windows

### OS Version

10

### ONNX Runtime Installation

Built from Source

### ONNX Runtime Version or Commit ID

1.13.1

### ONNX Runtime API

C++

### Architecture

X64

### Execution Provider

CUDA

### Execution Provider Library Version

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the minimal C++ reproduction in the issue and verify the ONNX Runtime 1.13.1, CUDA 11.4, Windows 10, Visual Studio 2019, and x64 configuration. Compare behavior with and without AppendExecutionProvider_CUDA and determine where initialization stops; done requires identifying a reproducible cause or actionable diagnostic result.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.