winml perf: post-result Windows 0xC0000005 in Microsoft.Windows.AI.MachineLearning.dll teardown

Summary

On Windows, winml perf can complete every warmup and timed iteration, print and persist valid results, and then terminate with 0xC0000005 during native teardown. Windows Error Reporting identifies Microsoft.Windows.AI.MachineLearning.dll from windowsml==2.0.300 as the faulting module.

This reproduces on bundled CPU with a recipe-free current-main OWLv2 build and with both checked-in candidate precision recipes, so it is not caused by OWLv2 export input binding or a recipe field. It blocks an otherwise passing model-support Goal L1 because process exit remains authoritative.

Environment

• Windows, Intel Core i9-10900X
• winml-cli current main: 3a2b2f7db5e70bb1f9039dd9037697ef92b2bc2a
• windowsml==2.0.300
• Python 3.11.14
• bundled CPUExecutionProvider
• checkpoint: google/owlv2-base-patch16

The tested commit contains both:

• #1220 / eef88afb75fc3b534fd93841e9a899b691924e2a (windowsml.EpCatalog registration and atexit catalog.close())
• #1223 / 57c2cdf2381cc3a55f1ef065d247d838b3c6a0db (concurrent native-stderr drain)

#1223 prevents pipe deadlock but does not affect this teardown access violation.

Exact reproduction

Build without any recipe:

uv run winml build -m google/owlv2-base-patch16 -o temp/baseline_owlv2 --ep cpu --device cpu --no-analyze --no-optimize --no-quant --no-compile --rebuild
uv run winml perf -m temp/baseline_owlv2/model.onnx --ep cpu --device cpu

The same failure occurs for independently built CPU fp32 and fp16 artifacts:

uv run winml perf -m <fp32-model.onnx> --ep cpu --device cpu
uv run winml perf -m <fp16-model.onnx> --ep cpu --device cpu

Observed on three runs:

A bounded retry with the exact fp32 artifact and --iterations 1 --warmup 0 exits 0, and a one-iteration synthetic Relu ONNX benchmark also exits 0. The defect therefore appears lifecycle/load-sensitive rather than an inference or input-binding failure.

Windows crash evidence

Application Error event 1000 for both exact recipe runs:

• faulting application: python.exe 3.11.14150.1013
• faulting module: Microsoft.Windows.AI.MachineLearning.dll 2.0.300.41724
• exception code: 0xc0000005
• fault offset: 0x00000000000026f5
• module path: .venv\Lib\site-packages\windowsml\lib\Microsoft.Windows.AI.MachineLearning.dll

Both runs produce the same WER bucket: 53def927dd6c2a0e007b5c91617a7da5.

The command also logs Init provider bridge failed. before benchmarking, even though explicit bundled CPU was requested.

Suspected lifecycle surface

Current main lazily creates a process-wide windowsml.EpCatalog in src/winml/modelkit/ep_path.py and registers an atexit callback that calls catalog.close(). winml perf retains a PerfBenchmark/model/WinMLSession through reporting and relies on object destruction rather than an explicit command-finally session/catalog shutdown sequence.

The exact native instruction causing the fault still needs symbolized dump analysis, but the proximate classification is clear: post-result WindowsML provider/catalog/session teardown, not model execution. Please verify shutdown ordering and ensure every ORT session/provider object is destroyed before closing the WindowsML catalog/runtime. Also consider avoiding WindowsML catalog activation when the user explicitly requests bundled CPU.

Acceptance criteria

• The default commands above exit 0 after persisting results for both fp32 and fp16.
• No post-result 0xC0000005 appears in Windows Application Error/WER.
• Explicit bundled CPU does not activate an unnecessary WindowsML catalog/provider bridge, or the catalog/session teardown order is made safe.
• Add a subprocess regression test that checks the actual process exit after benchmark completion; validating printed/persisted results alone is insufficient.