microsoft / microsoft/WindowsML

Python: onnxruntime-windowsml access violation in InferenceSession when a catalog EP (NvTensorRTRTX) is selected; same EP works under plain onnxruntime 1.30.0

Open
#22 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
29
Forks
5
Avg merge
11h 1m
Merged PRs (30d)
2

Description

Summary

onnxruntime-windowsml crashes with a native access violation inside InferenceSession(...) whenever an execution provider obtained from the Windows ML ExecutionProviderCatalog (here NvTensorRTRTXExecutionProvider, EP package 2.30.43.0) is registered with ort.register_execution_provider_library and selected for the session. Registration itself succeeds and the EP shows up in get_ep_devices(); the crash is at session creation, on every model tried, via both add_provider_for_devices and set_provider_selection_policy(PREFER_GPU).

The same EP DLL works when hosted by plain onnxruntime==1.30.0 (and by onnxruntime-directml==1.24.4), so the EP and the driver are fine; the failure is specific to the onnxruntime-windowsml wheel.

Environment

  • Windows 11, build 10.0.26200, x64
  • NVIDIA GeForce RTX 5070 Laptop GPU, driver 610.88
  • Python 3.12.10 (python.org, unpackaged), fresh venv
  • wasdk-Microsoft.Windows.AI.MachineLearning==2.3.0 (pins onnxruntime-windowsml==1.25.2.202605110140)
  • wasdk-Microsoft.Windows.ApplicationModel.DynamicDependency.Bootstrap==2.3.0
  • Also reproduces with onnxruntime-windowsml==1.28.0.202607272323 installed over the pin
  • Windows App SDK runtime 2.3.x / 2.4.0 present; WindowsWorkload.WinMLShared.5.2 present
  • EP package: Microsoft.WinML.NVIDIA.TRT-RTX.EP.2_2.30.43.0_x64 (installed via ensure_ready_async(), status Success)
  • Model: SqueezeNet 1.0 opset 12 from the ONNX model zoo (validated/vision/classification/squeezenet/model/squeezenet1.0-12.onnx)

Repro

pip install "wasdk-Microsoft.Windows.AI.MachineLearning[all]" wasdk-Microsoft.Windows.ApplicationModel.DynamicDependency.Bootstrap onnxruntime-windowsml
python -X faulthandler repro.py squeezenet1.0-12.onnx

repro.py:

import os, sys, faulthandler; faulthandler.enable()
import onnxruntime as ort                      # crashes regardless of import order vs the bootstrap
from winui3.microsoft.windows.applicationmodel.dynamicdependency.bootstrap import initialize
CTX = initialize()
import winui3.microsoft.windows.ai.machinelearning as winml
p = [p for p in winml.ExecutionProviderCatalog.get_default().find_all_providers() if "NvTensorRT" in p.name][0]
assert int(p.ensure_ready_async().get().status) == 1
os.add_dll_directory(os.path.dirname(p.library_path))
ort.register_execution_provider_library(p.name, p.library_path)
nv = [d for d in ort.get_ep_devices() if d.ep_name == p.name]
print("registered:", [d.ep_name for d in ort.get_ep_devices()], flush=True)
so = ort.SessionOptions(); so.add_provider_for_devices(nv, {})   # policy PREFER_GPU crashes identically
s = ort.InferenceSession(sys.argv[1], sess_options=so)           # <- access violation here
print("session providers:", s.get_providers())

Observed

[W:onnxruntime:Default, onnxruntime_pybind_module.cc:45 onnxruntime::python::CreateOrtEnv] Init provider bridge failed.
registered: ['CPUExecutionProvider', 'DmlExecutionProvider', 'DmlExecutionProvider', 'NvTensorRTRTXExecutionProvider']
Windows fatal exception: access violation
Current thread ... (most recent call first):
  File "...\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 635 in _create_inference_session
  File "...\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 529 in __init__
  File "repro.py", line 14 in <module>

Exit code 0xC0000005. No Python traceback, no Windows Error Reporting entry.

With verbose ORT logging the last lines before the crash are the plugin EP's own creation messages:

[I:onnxruntime:, tensorrt_rtx_provider_factory.cc:802 ...CreateEpImpl] Creating Execution Provider
[I:onnxruntime:, tensorrt_rtx_execution_provider.cc:2098 ...] Plugin EP has been created with name NvTensorRTRTXExecutionProvider

Expected

A session using NvTensorRTRTXExecutionProvider, as happens with the same DLL under plain onnxruntime==1.30.0:

python ctrl.py squeezenet1.0-12.onnx
ort 1.30.0 providers ['NvTensorRTRTXExecutionProvider', 'CPUExecutionProvider'] output (1, 1000, 1, 1) finite True

(ctrl.py is the same script minus the bootstrap/catalog lines, registering the EP DLL by its absolute path.)

Notes that may help

  • The onnxruntime-windowsml wheel ships onnxruntime.dll and DirectML.dll in capi/ but no onnxruntime_providers_shared.dll, and every import prints Init provider bridge failed. Both wheels that work carry that DLL. Copying a 1.24.4 onnxruntime_providers_shared.dll next to the 1.25.2 runtime did not change the crash, so it may be a symptom rather than the cause.
  • ExecutionProvider.try_register() returns True but does not make the EP visible to the Python ORT env, as the docs say; register_execution_provider_library is what adds it.
  • library_path is empty until ensure_ready_async() has completed, even when ready_state is already NOT_READY rather than NOT_PRESENT.
  • CPU and DirectML sessions in the same wheel work normally (outputs bit-identical to onnxruntime-directml).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run repro.py with the listed Windows ML and onnxruntime-windowsml packages, starting at the InferenceSession call in onnxruntime_inference_collection.py. Compare the catalog-registered EP path with ctrl.py and the plain onnxruntime wheel, including the provider bridge messages. Done means the Windows ML wheel can create a session with NvTensorRTRTXExecutionProvider without an access violation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.