microsoft / microsoft/onnxruntime

[XNNPACK] There is no obvious improvement when xnnpack turns on multithreading

Open
#13,819 15 comments 0 reactions 0 assignees View on GitHub
feature request platform:web
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

[profiling.zip](https://github.com/microsoft/onnxruntime/files/10138007/profiling.zip)
### Describe the issue

There is no obvious improvement when xnnpack turns on multithreading.
Under the same number of threads, xnnpack will not be better than wasm.

I'm not sure if this has something to do with
kOrtSessionOptionsConfigAllowIntraOpSpinning
and
intra_op_num_threads

### To reproduce

executionProviders: ['xnnpack'],
ort.env.wasm.numThreads = 4;

### Urgency

_No response_

### ONNX Runtime Installation

Built from Source

### ONNX Runtime Version or Commit ID

master

### Execution Provider

Other / Unknown

Contributor guide

Open the contributing guide

Research direction

Start with profiling.zip and reproduce the reported configuration using executionProviders: ['xnnpack'] and ort.env.wasm.numThreads = 4. Inspect how kOrtSessionOptionsConfigAllowIntraOpSpinning and intra_op_num_threads affect the comparison, then establish a measurable XNNPACK-versus-WASM result and identify the cause of the missing improvement.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, wasm
Domain
backend, machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.