microsoft / microsoft/onnxruntime
CUDA plugin EP GetEpDevices() finds no GPU on Jetson
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
On Jetson (integrated/platform-bus GPU, not PCI-attached), the CUDA plugin EP loads and creates its factory successfully, but Env::GetEpDevices() never returns a CUDA device, only CPUExecutionProvider. Root cause looks to be DetectGpuSysfsPaths in [device_discovery.cc](
https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/platform/linux/device_discovery.cc#L89), which skips any /sys/class/drm/cardN lacking a device/vendor sysfs attribute:
```cpp
if (!vendor_path_exists) {
LOGS_DEFAULT(VERBOSE) << "Skipping non-PCI DRM card: " << dir_item_path;
continue;
}
```
This assumes PCI-attached GPUs (comment references AMD XCP partitioning). Jetson's iGPU is on the platform/host1x bus, has no device/vendor file, and is unconditionally skipped. Fallback PCI-bus scanning then only finds the ARM host bridge.
## Verbose log
```
[V:onnxruntime, device_discovery.cc:89 DetectGpuSysfsPaths] Skipping non-PCI DRM card: "/sys/class/drm/card1"
[V:onnxruntime, device_discovery.cc:89 DetectGpuSysfsPaths] Skipping non-PCI DRM card: "/sys/class/drm/card0"
[V:onnxruntime, device_discovery.cc:308 GetGpuDevices] No GPUs found via /sys/class/drm. Falling back to PCI bus scanning via /sys/bus/pci/devices.
[I:onnxruntime, device_discovery_common.cc:34 operator()] Discovered OrtHardwareDevice {vendor_id:0x13b5, device_id:0x0, vendor:ARM, type:0, metadata: []}
[I:onnxruntime, cuda_plugin_ep.cc:93 CreateEpFactories] CreateEpFactories: Initializing CUDA Plugin EP with registration name: CUDAExecutionProvider
[I:onnxruntime, cuda_plugin_ep.cc:115 CreateEpFactories] CreateEpFactories: Successfully created CUDA EP factory
```
GetEpDevices() subsequently returns only CPUExecutionProvider / Microsoft.
## Expected behavior
GetEpDevices() should surface the Jetson integrated GPU, or discovery should have a fallback (e.g. platform/host1x bus check) alongside the PCI check for non-PCI-attached GPUs.
### To reproduce
```cpp
env.RegisterExecutionProviderLibrary(
"CUDAExecutionProvider",
ORT_TSTR("/opt/onnxruntime-plugins/cuda/libonnxruntime_providers_cuda.so")
);
auto ep_devices = env.GetEpDevices();
for (const auto & dev : ep_devices) {
if (std::string(dev.EpName()) == "CUDAExecutionProvider") {
sessionOptions.AppendExecutionProvider_V2(env, std::vector{dev}, Ort::KeyValuePairs{});
break;
}
}
````
### Urgency
_No response_
### Platform
Linux
### OS Version
Ubuntu 24.04.5
### ONNX Runtime Installation
Released Package
### ONNX Runtime Version or Commit ID
1.29.1
### ONNX Runtime API
C++
### Architecture
ARM64
### Execution Provider
CUDA
### Execution Provider Library Version
CUDA EP Plugin 0.1.0
Contributor guide
Assessment
This issue has not been assessed yet.