microsoft / microsoft/onnxruntime

Could not find an implementation for Relu(14)

Open
#11,725 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 8h
Merged PRs (30d)
179

Description

Describe the bug
I converted a TensorFlow/Keras model that internally uses a BERT model plus its corresponding tokenizer from TensorFlow Hub (bert_multi_cased_L-12_H-768_A-12) to ONNX using tf2onnx v1.9.3 (I tried v1.10.1, too) and ONNX Opset 14. To do so, I had to specify 2 extra opsets: ai.onnx.ml:1 and ai.onnx.contrib:1.

Loading the model and checking via

onnx_model = onnx.load(<my onnx model path>)
onnx.checker.check_model(onnx_model, full_check=True)

works as expected (i.e., the check passes without issues).

However, creating an inference session fails with

onnxruntime.capi.onnxruntime_pybind11_state.NotImplemented: [ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Could not find an implementation for Relu(14) node with name 'model/preprocessor/packer/PartitionedCall/PartitionedCall/PartitionedCall/PartitionedCall/Relu_1'

I tried other ONNX Opsets, too:

  • I couldn't try ONNX Opset 16 yet, because this isn't supported by tf2onnx yet.
  • ONNX Opset 15 fails as described above.
  • ONNX Opset 13 fails because the Relu recieves as input an int64 tensor at some point, which is only supported from ONNX Opset 14 on.

Do you have any idea of what could possibly go wrong?

Urgency
This blocks a proprietary project.

System information

  • OS Platform and Distribution: Ubuntu 20.04.4 LTS
  • ONNX Runtime installed from (source or binary): binary
  • ONNX Runtime version: v1.10.0 (tried v1.11.1, too)
  • ONNX Runtime extensions version: 0.4.2
  • ONNX version: v1.10.0
  • Python version: v3.9
  • Visual Studio version (if applicable): --
  • GCC/Compiler version (if compiling from source): --
  • CUDA/cuDNN version: CUDA Version: 11.4 (shouldn't matter; doesn't work on CPU or GPU)
  • GPU model and memory: NVIDIA RTX 2060, 6GByte

To Reproduce
I'm happy to share a non-trained ONNX model that exhibits the described behavior (it's 800 MByte, tho). I can share the TensorFlow/Keras model I created the ONNX model from, too, but this is even larger.

To reproduce, just create an inference session using the ONNX model:

import onnxruntime as ort
from onnxruntime_extensions import get_library_path

session_options = ort.SessionOptions()
session_options.register_custom_ops_library(get_library_path())
session_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_EXTENDED
return ort.InferenceSession(
    onnx_model_filename,
    providers=['CPUExecutionProvider'],
    sess_options=session_options
)

Expected behavior
Relu(14) implementation should be found by the ONNX Runtime.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Python reproduction using ort.InferenceSession, CPUExecutionProvider, and the registered onnxruntime_extensions library, then inspect the supplied ONNX model's Relu node and opset information. Reproduce the failure with ONNX Runtime 1.10.0 or 1.11.1; done means identifying a minimal reproducible model or a documented runtime incompatibility and validating the result across the reported opsets.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.