TensorRT 10.16 Myelin crash on Blackwell (sm_120) — Broadcast stride mismatch in SoVITS decoder
Open
Nobody has claimed this yet.
Module:Engine Build
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Environment
- GPU: RTX 5080 (Blackwell, sm_120, 16GB)
- Driver: 596.21 / CUDA 13.2
- TensorRT: 10.16.1.11 (pip + SDK)
- ONNX exported from GPT-SoVITS v2ProPlus (opset 17)
Error
Building sovits.engine with trtexec --fp16 crashes with:
Internal Error: MyelinCheckException:
CHECK_EQ(dim_count(), stride_order().size()) failed. LHS: 4 RHS: 3
Could not find any implementation for node
{ForeignNode[vq_model.ge_to512.bias + ONNXTRT_Broadcast_270.../dec/Tanh]}
All 7 other engines build fine. Only sovits.onnx fails.
Root Cause
8 Add nodes in the ONNX graph have 1D constant bias inputs
(shape=[128], [512], [1024]) broadcast-added to higher-dimensional
tensors. Myelin fuses these into ForeignNodes but infers stride as
3D when 4D is expected.
Workaround
Reshaping biases to [1,1,N] via ONNX GraphSurgeon allows compilation
but adds inference overhead. Pure ONNX FP16 inference is faster than
the TRT workaround (RTF 0.34 vs 0.47).
Reproduction
All scripts, patches, and models attached in repro.zip.
See repro/README.md for step-by-step instructions.
https://drive.google.com/file/d/1V-n3Xco_jBafDdaAmTsRQR_u1O3bawHP/view?usp=sharing
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with repro/README.md and reproduce the sovits.onnx build using trtexec. Inspect the eight Add nodes with 1D bias inputs and the failing ForeignNode around vq_model.ge_to512.bias, comparing them with the GraphSurgeon workaround. Done means the original graph builds successfully without the Myelin stride-mismatch crash.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100