microsoft / microsoft/onnxruntime

[Feature Request] [Node.js] - Support for onnxruntime-extensions operations when using onnxruntime-node

Open
#18,587 0 comments 0 reactions 0 assignees View on GitHub
feature request platform:web
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

Today the currently vended `onnxruntime-node` from NPM is unable to operate on .onnx models that leverage `onnxruntime-extensions`. A good example of this is when attempting to use OpenAI's Whisper model. This model functions fine when using `onnxruntime-web` or when using [onnxruntime-react-native](https://github.com/microsoft/onnxruntime/blob/main/js/react_native/onnxruntime-react-native.podspec#L30-L38) however when attempting to use operations provided by extensions in `onnxruntime-node` we run into issues.

Example
---
Model: `whisper_cpu_int8_0_model.onnx`
Quantized: Yes (8-Bit)
Runtime: onnxruntime-node - 1.17.0 (Precompiled binaries from NPM)
Platform: Linux - 5.15.133.1-microsoft-standard-WSL2
```
/home/ian/onnxruntime/js/node/dist/backend.js:27
__classPrivateFieldGet(this, _OnnxruntimeSessionHandler_inferenceSession, "f").loadModel(pathOrBuffer.buffer, pathOrBuffer.byteOffset, pathOrBuffer.byteLength, options);
^

Error: Fatal error: ai.onnx.contrib:BpeDecoder(-1) is not a registered function/op
at new OnnxruntimeSessionHandler (/home/ian/onnxruntime/js/node/dist/backend.js:27:92)
at /home/ian/onnxruntime/js/node/dist/backend.js:64:29
at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
```

It seems like the current CMake files for `onnxruntime-node` simply don't include the build options to enable `onnxruntime-extensions` it would be great if we could include these in the node build as well so I can stop using `onnxruntime-web` in NodeJS and degrading performance.

### Describe scenario use case

Being able to use `onnxruntime-node` to execute models that leverage operations provided by `onnxruntime-extensions` this includes a wide variety of models.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the CMake files used to build onnxruntime-node and compare their options with the onnxruntime-web and onnxruntime-react-native builds. Reproduce the failure with the Whisper model and the BpeDecoder error, then verify that onnxruntime-node can execute models using onnxruntime-extensions operations.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, node.js
Domain
build-system, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.