NeMo fastpitch onnx convert to tensorrt failure of TensorRT 10.3.0

Description

Environment

I'm using this docker image: nvcr.io/nvidia/tensorrt:24.08-py3

TensorRT Version:

NVIDIA GPU: L40S

NVIDIA Driver Version: 535.183.01

CUDA Version: 12.6

CUDNN Version: 9.3.0

Operating System: Ubuntu22.04

Python Version (if applicable): 3.10.12

PyTorch Version (if applicable): 2.4.0

Relevant Files

Model link:

Steps To Reproduce

First install NeMo

pip install nemo_toolkit['tts']

1. Code used to generate the onnx model.

from nemo.collections.tts.models.fastpitch import FastPitchModel

spec_model = FastPitchModel.from_pretrained("tts_en_fastpitch")
spec_model.export('ljspeech.onnx', onnx_opset_version=20)

2. command that reproduce the error

trtexec --onnx=ljspeech.onnx --minShapes=text:1x32,pitch:1x32,pace:1x32 --optShapes=text:1x768,pitch:1x768,pace:1x768 --maxShapes=text:1x1664,pitch:1x1664,pace:1x1664 --shapes=text:1x768,pitch:1x768,pace:1x768 --memPoolSize=workspace:4096 --noTF32 --saveEngine=ljspeech.engine

3. The error is:

[E] Error[7]: IExecutionContext::enqueueV3: Error Code 7: Internal Error (/decoder/layers.0/dec_attn/MatMul_1: attempt to multiply two matrices with mismatching dimensions Condition '==' violated: 0 != 1. Instruction: CHECK_EQUAL 0 1.)
[E] Error occurred during inference

Commands or scripts:

Have you tried the latest release?: yes

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): yes

Start with the FastPitchModel.export call that produces ljspeech.onnx, then run the provided trtexec command and compare its TensorRT failure with the successful ONNX Runtime execution. Inspect the /decoder/layers.0/dec_attn/MatMul_1 shape mismatch and determine the relevant TensorRT compatibility boundary. Done means the supplied model and dynamic shapes execute successfully in TensorRT 10.3.0.