microsoft / microsoft/onnxruntime
[Feature Request] [Node.js] - Support for onnxruntime-extensions operations when using onnxruntime-node
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
Today the currently vended `onnxruntime-node` from NPM is unable to operate on .onnx models that leverage `onnxruntime-extensions`. A good example of this is when attempting to use OpenAI's Whisper model. This model functions fine when using `onnxruntime-web` or when using [onnxruntime-react-native](https://github.com/microsoft/onnxruntime/blob/main/js/react_native/onnxruntime-react-native.podspec#L30-L38) however when attempting to use operations provided by extensions in `onnxruntime-node` we run into issues.
Example
---
Model: `whisper_cpu_int8_0_model.onnx`
Quantized: Yes (8-Bit)
Runtime: onnxruntime-node - 1.17.0 (Precompiled binaries from NPM)
Platform: Linux - 5.15.133.1-microsoft-standard-WSL2
```
/home/ian/onnxruntime/js/node/dist/backend.js:27
__classPrivateFieldGet(this, _OnnxruntimeSessionHandler_inferenceSession, "f").loadModel(pathOrBuffer.buffer, pathOrBuffer.byteOffset, pathOrBuffer.byteLength, options);
^
Error: Fatal error: ai.onnx.contrib:BpeDecoder(-1) is not a registered function/op
at new OnnxruntimeSessionHandler (/home/ian/onnxruntime/js/node/dist/backend.js:27:92)
at /home/ian/onnxruntime/js/node/dist/backend.js:64:29
at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
```
It seems like the current CMake files for `onnxruntime-node` simply don't include the build options to enable `onnxruntime-extensions` it would be great if we could include these in the node build as well so I can stop using `onnxruntime-web` in NodeJS and degrading performance.
### Describe scenario use case
Being able to use `onnxruntime-node` to execute models that leverage operations provided by `onnxruntime-extensions` this includes a wide variety of models.
Contributor guide
Research direction
Start by inspecting the CMake files used to build onnxruntime-node and compare their options with the onnxruntime-web and onnxruntime-react-native builds. Reproduce the failure with the Whisper model and the BpeDecoder error, then verify that onnxruntime-node can execute models using onnxruntime-extensions operations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, node.js
- Domain
- build-system, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100