microsoft / microsoft/onnxruntime
Could not find an implementation for Relu(14)
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 179
Description
Describe the bug
I converted a TensorFlow/Keras model that internally uses a BERT model plus its corresponding tokenizer from TensorFlow Hub (bert_multi_cased_L-12_H-768_A-12) to ONNX using tf2onnx v1.9.3 (I tried v1.10.1, too) and ONNX Opset 14. To do so, I had to specify 2 extra opsets: ai.onnx.ml:1 and ai.onnx.contrib:1.
Loading the model and checking via
onnx_model = onnx.load(<my onnx model path>)
onnx.checker.check_model(onnx_model, full_check=True)
works as expected (i.e., the check passes without issues).
However, creating an inference session fails with
onnxruntime.capi.onnxruntime_pybind11_state.NotImplemented: [ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Could not find an implementation for Relu(14) node with name 'model/preprocessor/packer/PartitionedCall/PartitionedCall/PartitionedCall/PartitionedCall/Relu_1'
I tried other ONNX Opsets, too:
- I couldn't try ONNX Opset 16 yet, because this isn't supported by tf2onnx yet.
- ONNX Opset 15 fails as described above.
- ONNX Opset 13 fails because the Relu recieves as input an int64 tensor at some point, which is only supported from ONNX Opset 14 on.
Do you have any idea of what could possibly go wrong?
Urgency
This blocks a proprietary project.
System information
- OS Platform and Distribution: Ubuntu 20.04.4 LTS
- ONNX Runtime installed from (source or binary): binary
- ONNX Runtime version: v1.10.0 (tried v1.11.1, too)
- ONNX Runtime extensions version: 0.4.2
- ONNX version: v1.10.0
- Python version: v3.9
- Visual Studio version (if applicable): --
- GCC/Compiler version (if compiling from source): --
- CUDA/cuDNN version: CUDA Version: 11.4 (shouldn't matter; doesn't work on CPU or GPU)
- GPU model and memory: NVIDIA RTX 2060, 6GByte
To Reproduce
I'm happy to share a non-trained ONNX model that exhibits the described behavior (it's 800 MByte, tho). I can share the TensorFlow/Keras model I created the ONNX model from, too, but this is even larger.
To reproduce, just create an inference session using the ONNX model:
import onnxruntime as ort
from onnxruntime_extensions import get_library_path
session_options = ort.SessionOptions()
session_options.register_custom_ops_library(get_library_path())
session_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_EXTENDED
return ort.InferenceSession(
onnx_model_filename,
providers=['CPUExecutionProvider'],
sess_options=session_options
)
Expected behavior
Relu(14) implementation should be found by the ONNX Runtime.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Python reproduction using ort.InferenceSession, CPUExecutionProvider, and the registered onnxruntime_extensions library, then inspect the supplied ONNX model's Relu node and opset information. Reproduce the failure with ONNX Runtime 1.10.0 or 1.11.1; done means identifying a minimal reproducible model or a documented runtime incompatibility and validating the result across the reported opsets.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100