microsoft / microsoft/onnxruntime

[Performance] Inference time much longer when using JavaScript than when using Python

Open
#14,220 4 comments 0 reactions 1 assignee Claimed by @shalvamist View on GitHub
api:Javascript
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

Hi everyone,

I custom-trained a face dectection algorithm using YOLO and exported it to the ONNX format.
- When I perform inferences with the ONNX file on Python, the inference time is around 0.08s and the time needed to create an inference session is around 0.10s.
- When I perform inferences with the ONNX file on JavaScript, the inference time is around 0.6s and the time needed to create an inference session is around 0.20s.

I face the same issue when running other ONNX models (my inference time on Javascript is always significantly longer than on Python).
Could you please guide me to improve my inference time under Javascript ? Thank you in advance !

### To reproduce

Run a same ONNX file with Python and Javascript. My code to make on inference on Javascript is inspired from the tutorial provided on the ONNX website (https://onnxruntime.ai/docs/tutorials/web/classify-images-nextjs-github-template.html).

### Urgency

I need to address this issue ASAP to be able to keep building my Electron app functionalities.

### Platform

Windows

### OS Version

10

### ONNX Runtime Installation

Other / Unknown

### ONNX Runtime Version or Commit ID

1.13

### ONNX Runtime API

JavaScript

### Architecture

X64

### Execution Provider

Default CPU

### Execution Provider Library Version

_No response_

### Model File

GitHub does not support the ONNX file format.

### Is this a quantized model?

Unknown

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.