NVIDIA / NVIDIA/TensorRT

failed to build the serialized network for a simple model with only Or and Cast: Internal Error (Could not find any implementation for node)

Open
#4,511 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Module:Engine Build Module:ONNX Module:Runtime
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

For the following simple onnx model,
Image
the results produced by onnxruntime are as follows:

 [array([[[[1.]],

        [[1.]],

        [[1.]]]], dtype=float32)]

However, when I run it using tensorrt, an internal error occurred as follows:

Internal Error: MyelinCheckException: host_instr.cpp:2806: CHECK(0) failed. 
[07/06/2025-10:30:33] [TRT] [E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [myelin_graph.h:attachExceptionMsgToGraph:1139] MyelinCheckException: host_instr.cpp:2806: CHECK(0) failed. 
[07/06/2025-10:30:33] [TRT] [E] IBuilder::buildSerializedNetwork: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[or_constant_out + ONNXTRT_Broadcast...node_of_output]}.)

Environment

TensorRT Version: 10.12.0.36

NVIDIA GPU: GeForce RTX 3080

NVIDIA Driver Version: 535.183.01

CUDA Version: 12.2

CUDNN Version: none

Operating System: ubuntu 20.04

Python Version (if applicable): 3.12.9

Relevant Files

Model link:

Steps To Reproduce

This issue can be reproduced by the following code with the model in the attachment.

from typing import Dict, List, Literal, Optional
import sys
import os

import numpy as np
import onnx
import onnxruntime

import tensorrt as trt

import argparse
import pickle

def test():
    onnx_model = onnx.load('222.onnx')
    
    with open("inputs.pkl", "rb") as fp:
        inputs = pickle.load(fp)

    try:
        ort_session = onnxruntime.InferenceSession(
            onnx_model.SerializeToString(), providers=["CPUExecutionProvider"]
        )
        ort_output = ort_session.run([], inputs)
    except Exception as e:
        print(e)
        print("This model cannot be executed by onnxruntime!")
        sys.exit(1)
    
    print("ONNXRuntime:\n", ort_output)
    
    #--------------------------------------------------------
        
    trt_logger = trt.Logger(trt.Logger.WARNING)
    trt.init_libnvinfer_plugins(trt_logger, '')
    builder = trt.Builder(trt_logger)
    #network = builder.create_network()
    network = builder.create_network(flags=1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))

    parser = trt.OnnxParser(network, trt_logger)
    with open('222.onnx', 'rb') as model_file:
        if not parser.parse(model_file.read()):
            for error in range(parser.num_errors):
                print(parser.get_error(error))
            sys.exit(1)
    
    config = builder.create_builder_config()
    serialized_engine = builder.build_serialized_network(network, config)
    
if __name__ == "__main__":
    test()
    

testcase.zip

Commands or scripts:

Have you tried the latest release?: yes

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): the mode can be executed by onnxruntime.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the Python reproduction with testcase.zip, including 222.onnx and inputs.pkl, and compare the ONNXRuntime result with TensorRT's build_serialized_network failure. Inspect the parser and builder output around the Or and Cast nodes. Done means identifying why no implementation is selected or documenting a confirmed fix with a successful serialized network build.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.