NVIDIA / NVIDIA/TensorRT

TensorRT 10.16 Myelin crash on Blackwell (sm_120) — Broadcast stride mismatch in SoVITS decoder

Open
#4,743 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Module:Engine Build
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Environment
- GPU: RTX 5080 (Blackwell, sm_120, 16GB)
- Driver: 596.21 / CUDA 13.2
- TensorRT: 10.16.1.11 (pip + SDK)
- ONNX exported from GPT-SoVITS v2ProPlus (opset 17)

Error
Building sovits.engine with trtexec --fp16 crashes with:


Internal Error: MyelinCheckException:
  CHECK_EQ(dim_count(), stride_order().size()) failed.  LHS: 4  RHS: 3

Could not find any implementation for node
  {ForeignNode[vq_model.ge_to512.bias + ONNXTRT_Broadcast_270.../dec/Tanh]}


All 7 other engines build fine. Only sovits.onnx fails.

Root Cause
8 Add nodes in the ONNX graph have 1D constant bias inputs
(shape=[128], [512], [1024]) broadcast-added to higher-dimensional
tensors. Myelin fuses these into ForeignNodes but infers stride as
3D when 4D is expected.

Workaround
Reshaping biases to [1,1,N] via ONNX GraphSurgeon allows compilation
but adds inference overhead. Pure ONNX FP16 inference is faster than
the TRT workaround (RTF 0.34 vs 0.47).

Reproduction
All scripts, patches, and models attached in repro.zip.
See repro/README.md for step-by-step instructions.

https://drive.google.com/file/d/1V-n3Xco_jBafDdaAmTsRQR_u1O3bawHP/view?usp=sharing

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with repro/README.md and reproduce the sovits.onnx build using trtexec. Inspect the eight Add nodes with 1D bias inputs and the failing ForeignNode around vq_model.ge_to512.bias, comparing them with the GraphSurgeon workaround. Done means the original graph builds successfully without the Myelin stride-mismatch crash.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.