microsoft / microsoft/onnxruntime

UnregisterExecutionProviderLibrary can access torn-down CRT state during Windows process shutdown

Open
#32,402 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

platform:windows
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

Calling `OrtApi::UnregisterExecutionProviderLibrary` from late process-shutdown cleanup can crash inside ONNX Runtime.

ONNX Runtime already detects Windows process termination in `onnxruntime/core/dll/dllmain.cc`: when `DLL_PROCESS_DETACH` has a non-null reserved parameter, it sets `g_is_shutting_down`. `OrtEnv::Release` uses this state to intentionally skip destruction during process termination

However, `OrtApi::UnregisterExecutionProviderLibrary` does not check that state before accessing `OrtEnv::GetEnvironment()`. A consumer that explicitly unregisters an EP from global or DLL teardown can therefore enter `Environment::UnregisterExecutionProviderLibrary` after CRT state has begun shutting down.

This occurs across all EPs

The original failure is:

- Exception: `0xC0000005` access violation
- Faulting code: `onnxruntime!std::ctype::widen`
- Invalid read address: `0xFFFFFFFFFFFFFFFF`
- AppVerifier state included `0xF0F0F0F0F0F0F0F0`, consistent with invalid or poisoned teardown state
- Call path:
`onnxruntime::Environment::UnregisterExecutionProviderLibrary`
-> consumer execution-provider cleanup
-> global destructor
-> `LdrShutdownProcess`

I have included a potential fix PR here: https://github.com/microsoft/onnxruntime/pull/32401

### To reproduce

1. Build or use ONNX Runtime as a Windows shared library.
2. Create an `OrtEnv`.
3. Dynamically register an execution-provider library using `RegisterExecutionProviderLibrary`.
4. Retain the environment and registration name in a consumer DLL or global owner.
5. From that owner's destructor, call `UnregisterExecutionProviderLibrary`.
6. Arrange for ONNX Runtime to receive `DLL_PROCESS_DETACH` for process termination before the consumer's global destructor runs. This can occur when ONNX Runtime is delay-loaded by the consumer.
7. Exit the process with Application Verifier enabled.

### Urgency

_No response_

### Platform

Windows

### OS Version

26683.1002

### ONNX Runtime Installation

Built from Source

### ONNX Runtime Version or Commit ID

1.28.1

### ONNX Runtime API

C++

### Architecture

X64

### Execution Provider

Other / Unknown

### Execution Provider Library Version

_No response_

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read onnxruntime/core/dll/dllmain.cc for g_is_shutting_down, then trace OrtEnv::Release and OrtApi::UnregisterExecutionProviderLibrary into Environment::UnregisterExecutionProviderLibrary. Reproduce the described Windows shared-library teardown with Application Verifier and verify that late unregister cleanup no longer accesses torn-down CRT state.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.