"Device to shape host node should not be folded into myelin" failure of TensorRT 10.5 when running trtexec on GPU L4

Description

I am trying to compile our model with trt, and failed. I locate the problem to our embedding layer. We are using torch.nn.EmbeddingBag. So I create a mini model with it and export it to onnx in torchscript way. Then, I run trtexec with the exported onnx graph, and got this error

[10/18/2024-18:35:44] [E] Error[2]: [myelinBuilderUtils.cpp::getMyelinSupportType::1087] Error Code 2: Internal Error (Assertion result != MyelinSupportType::kREQUIRES_MYELIN || n.getType() != NodeType::kDEVICE_TO_SHAPE_HOST failed. Device to shape host node should not be folded into myelin.)

If I export the model in dynamo way, then I will see error

[10/18/2024-18:51:46] [E] [TRT] ModelImporter.cpp:942: ERROR: onnxOpCheckers.cpp:993 In function checkSequenceEmpty: [8] false [10/18/2024-18:51:46] [E] [TRT] ModelImporter.cpp:942: ERROR: onnxOpCheckers.cpp:901 In function checkConcatFromSequence: [8] false

For the torchscript onnx graph, I have tried to use polygraph to bisect where it is wrong, it seems there's something wrong with the loop subgraph, but I don't know which node in the loop triggers that "Device to shape host node" error.

Can anyone help take a look please? or share any tools that I can dive deep? Basically torch.nn.EmbeddingBag should be supported very smoothly, right? I wonder how other people deal with this layer. Thanks.

Environment

TensorRT Version: 10.5

NVIDIA GPU: L4

NVIDIA Driver Version: 560.35.03

CUDA Version: 12.6

CUDNN Version: 9.5

Operating System: Ubuntu 24.04

Python Version (if applicable): 3.11.9

Tensorflow Version (if applicable):

PyTorch Version (if applicable): 2.5.0

Baremetal or Container (if so, version):

Relevant Files

Repo artifacts: demo.zip

It contains:

• a notebook about how I set up the mini model, and export it to onnx, and trtexec it
• two exported onnx graphs, in torchscript way and in dynamo way
• a onnx data file, when exported in dynamo way

Steps To Reproduce

Commands or scripts:

unzip the demo.zip, run the embedding_bag_repo.ipynb notebook

or you can run trtexec with the onnx file I uploaded in the zip file

trtexec --onnx=demo_embedding_bag_dynamo.onnx
trtexec --onnx=demo_embedding_bag_ts.onnx

Have you tried the latest release?:

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):

Start with demo.zip and embedding_bag_repo.ipynb; run the two trtexec commands against the supplied ONNX graphs on the stated TensorRT 10.5 environment. Compare the torchscript and dynamo failures and use the included graphs and data with Polygraphy to isolate the loop node. Done means the minimal failing path is identified and the reported error is resolved or clearly documented.