microsoft / microsoft/onnxruntime

WebGPU EP crashes on macOS when running concurrent multi threaded inference

Open
#31,627 4 comments 0 reactions 0 assignees View on GitHub
ep:CoreML ep:WebGPU platform:web
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

**Note:** Re opening this issue as a follow up to #27592, which was automatically closed by the stale bot despite being an active (and I can't click to re open), reproducible bug. Confirmed still present on latest ORT (`v1.28`) so wanted to create a new anyway.

The WebGPU Execution Provider crashes on macOS (Apple Silicon) when multiple threads create sessions and run inference concurrently. The crash is non-deterministic and manifests as different errors on each run, indicating a race condition in Dawn's Metal backend.

Observed crashes (non-deterministic, varies per run)
Run 1 — Metal assertion:
```
**-[AGXG15GFamilyCommandBuffer blitCommandEncoderCommon:]:832:**
failed assertion `A command encoder is already encoding to this command buffer'
```

**Run 2 — Memory corruption:**
` malloc: *** error for object 0x784797e90: pointer being freed was not allocated
`

**Run 3 — Trace trap:**
` [1] 56040 trace trap cargo run --release -- version-RFB-640.onnx`

Notes:
- Single-threaded session creation and inference works fine - Sequential (non-concurrent) multi-session usage works fine
- The same workload works correctly with the CoreML EP

### To reproduce

minimal Repo: https://github.com/altunenes/ort-webgpu-thread-crash

git clone https://github.com/altunenes/ort-webgpu-thread-crash.git
cargo run --release -- version-RFB-640.onnx
(simple face detection model)

The repro creates 2 persistent WebGPU sessions, then spawns 5 threads that each create a session and run inference concurrently while the main thread also runs inference on the persistent sessions.

### Urgency

_No response_

### Platform

Mac

### OS Version

26.2

### ONNX Runtime Installation

Built from Source

### ONNX Runtime Version or Commit ID

1.28

### ONNX Runtime API

Other / Unknown

### Architecture

X64

### Execution Provider

Other / Unknown

### Execution Provider Library Version

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the linked minimal repository and run `cargo run --release version-RFB-640.onnx` on macOS to reproduce the concurrent WebGPU session crash. Compare the concurrent workload with single-threaded and sequential multi-session runs, then verify that concurrent inference completes without the reported Metal assertions, memory errors, or trace traps.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, macos, rust
Domain
backend, machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.