microsoft / microsoft/onnxruntime

Running onnxruntime with a gpu in nodejs?

Open
#14,127 5 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ep:CUDA feature request platform:web platform:windows
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

Hello!

I'm trying to run yolov7 in nodejs but I am getting poor performance as it is using my cpu instead of my gpu.
Onnxruntime-web mentions the capability of using nodejs and gpu but when requiring it in node.js I get "Uncaught Error Error"
How would I get onnxruntime-node to run on my gpu for better performance?

Thanks!

### To reproduce

I'm using yolov7-w6-pose-nms.onnx from https://github.com/trancongman276/yolov7-pose

### Urgency

_No response_

### Platform

Windows

### OS Version

Windows 10

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

onnxruntime-node@1.13.1

### ONNX Runtime API

JavaScript

### Architecture

X64

### Execution Provider

CUDA

### Execution Provider Library Version

_No response_

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue on Windows 10 with onnxruntime-node@1.13.1, the yolov7-w6-pose-nms.onnx model, and the CUDA execution provider. Check why the Node.js package reports an error and uses the CPU; done means inference runs through the GPU, but the issue does not name a repository file or test to guide the investigation.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, machine-learning, node.js
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.