microsoft / microsoft/onnxruntime

[Performance] `import onnxruntime` takes ~9 s on Windows since 1.22.0 (was ~0.2 s in 1.21.1), no model involved

Open
#32,220 1 comment 0 reactions 0 assignees View on GitHub
platform:windows
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

Importing the package — no model, no `InferenceSession`, nothing else — takes **8.6–11.5 s** on Windows
starting with 1.22.0. Up to and including 1.21.1 the same import takes **~0.2 s**. The regression is
reproducible on a clean virtualenv containing nothing but `onnxruntime`, and is stable across repeated
cold process starts (not a first-run/file-cache effect).

`python -X importtime` puts effectively all of it inside the native module init:

```
9108800 onnxruntime
9103847 onnxruntime.capi._pybind_state
9030855 onnxruntime.capi.onnxruntime_pybind11_state <-- 9.03 s of 9.11 s
123178 numpy._core._multiarray_umath
```

The Python-level package code does nothing heavy — `_pybind_state.py` only does
`from . import _ld_preload`.

### Bisect

| onnxruntime | `import onnxruntime` (cold process, repeated) |
|---|---|
| 1.20.1 | 202 / 240 ms |
| **1.21.1** | **212 / 268 ms** — last good |
| **1.22.0** | **11 525 / 11 833 ms** — first bad |
| 1.25.0 | 8 892 / 9 080 ms |
| 1.27.0 | 8 735 / 8 797 ms |
| 1.28.0 | 8 604 / 9 276 ms |
| 1.29.0 | 8 601 / 9 162 ms |

(1.23.x and 1.26.x not tested; 1.24.0 is not on PyPI.)

### What this is *not*

Each of these was measured, not assumed:

| Hypothesis | Measurement | Verdict |
|---|---|---|
| Slow disk / large binary | raw `read_bytes()` of the 18 MB `onnxruntime.dll` | 4 ms — ruled out |
| DLL load / AV scan on image load | `ctypes.WinDLL()` on `onnxruntime.dll` **and** on `onnxruntime_pybind11_state.pyd` | 4 ms each — ruled out |
| Antivirus | Defender only, `EnableNetworkProtection: 0`, no third-party AV, clean Winsock catalog | ruled out |
| Network call during import | `socket.getaddrinfo` and `socket.socket.connect` instrumented across the import | 0 calls — ruled out |
| Slow machine | 6M-iteration Python loop: 660 ms | ruled out |
| Model/graph cost | `InferenceSession()` on a real 128 MB BGE-small ONNX model: 179 ms (1.20.1) / 228 ms (1.21.1) / 198 ms (1.29.0); `run()` 5 ms in all | ruled out — session creation is fine in every version |

So the cost is neither the file nor the model: it is whatever the extension module's init does before
any user code runs.

### Suspicion (unconfirmed)

The 1.21.1 → 1.22.0 boundary lines up with the **Auto EP Selection Infrastructure** shipped in v1.22
(#24430). Scanning UTF-16 strings in `onnxruntime_pybind11_state.pyd` (1.22.0) shows device-enumeration
strings that are consistent with DXGI adapter enumeration running as part of that infrastructure:

```
DxgiAdapterNumber
DxgiHighPerformanceIndex
DxgiVideoMemory
Discrete
```

This machine has hybrid graphics (AMD Radeon iGPU driving the display + an idle NVIDIA RTX 3060 Laptop
GPU), so "enumeration wakes the parked discrete GPU" was the obvious guess — **but I tested it and it is
wrong**: waking the dGPU with `nvidia-smi -L` first (1.9 s) and then importing still gives 9 638 ms and
9 206 ms. So device enumeration remains the prime suspect, but the specific stall inside it is not
something I can pin down without debug symbols.

I could not find an existing issue for this. I also could not find any environment switch to skip
auto-EP/device discovery at import — scanning the binary for `ORT_*` turns up only 16 strings, all
unrelated (arena allocator, graph optimization levels, attention flags).

### Why it matters

For a long-lived process 9 s at startup is invisible. For short-lived processes that import the package
per invocation — CLI tools, editor/agent hooks, serverless, test workers — it dominates everything else.
In my case a per-turn hook that embeds one short string cost ~10 s, of which ~9 s was this import;
pinning `onnxruntime==1.21.1` took the same hook to ~1.2 s with no other change.

### To reproduce

```powershell
py -3.13 -m venv ort-test
.\ort-test\Scripts\python.exe -m pip install "onnxruntime==1.21.1"
.\ort-test\Scripts\python.exe -c "import time; t=time.monotonic(); import onnxruntime; print(f'{(time.monotonic()-t)*1000:.0f} ms')"
# -> ~210 ms

.\ort-test\Scripts\python.exe -m pip install "onnxruntime==1.22.0"
.\ort-test\Scripts\python.exe -c "import time; t=time.monotonic(); import onnxruntime; print(f'{(time.monotonic()-t)*1000:.0f} ms')"
# -> ~11 500 ms
```

Run each a few times — the timing is stable, so it is not a cold-cache artifact.

### Urgency

No — there is a clean workaround (pin `onnxruntime<1.22`). Filing because it is a silent ~40x startup
regression that a short-lived-process user is unlikely to attribute to ONNX Runtime.

### Platform

Windows

### OS Version

Windows 11 Pro 24H2, build 26200 (x64).
AMD Ryzen 7 5800H (8C/16T), 32 GB RAM.
Hybrid graphics: AMD Radeon integrated (driver 31.0.12044.47003) + NVIDIA GeForce RTX 3060 Laptop GPU
(driver 32.0.16.1088).

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.22.0 first bad, 1.21.1 last good; also reproduced on 1.25.0, 1.27.0, 1.28.0, 1.29.0

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

Default CPU

### Execution Provider Library Version

n/a — the package is only imported; no session is created

### Model File

Not applicable — no model is loaded. (For the control measurement above I used
`onnx-community/bge-small-en-v1.5` ONNX, 128 MB, which loads in ~200 ms on every version tested.)

### Is this a quantized model?

No

Contributor guide

Open the contributing guide

Research direction

Reproduce the import timing with the provided PowerShell commands, then compare the native initialization path in 1.21.1 and 1.22.0, focusing on onnxruntime.capi.onnxruntime_pybind11_state and the Auto EP Selection Infrastructure from #24430. The Python-level entry point is _pybind_state.py, which only imports _ld_preload. Done means identifying and removing the import-time regression while preserving normal package import and session behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
machine-learning, operating-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.