ONNX conversion with lower performance engine
Open
Nobody has claimed this yet.
Module:Performance
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
I am using following codes to conversion ONNX model:
trtexec --verbose --onnx=./resnet50-v1-12.onnx --fp16 \
--minShapes=data:1x3x224x224 \
--optShapes=data:32x3x224x224 \
--maxShapes=data:1024x3x224x224 \
--saveEngine=./resnet50_fp16-verbose.trt --best
in docker image nvcr.io/nvidia/tensorrt:25.08-py3
on H200
[09/09/2025-11:19:31] [V] [TRT] Engine Layer Information:
Layer(Reformat): Reformatting CopyNode for Input Tensor 0 to resnetv17_conv0_fwd + resnetv17_batchnorm0_fwd + resnetv17_relu0_fwd + resnetv17_pool0_fwd, Tactic: 0x00000000000003ea, data (Float[-1,3,224,224]) -> Reformatted Input Tensor 0 to resnetv17_conv0_fwd + resnetv17_batchnorm0_fwd + resnetv17_relu0_fwd + resnetv17_pool0_fwd (Int8[-1,3,224,224])
Layer(CaskConvActPool): resnetv17_conv0_fwd + resnetv17_batchnorm0_fwd + resnetv17_relu0_fwd + resnetv17_pool0_fwd, Tactic: 0x6ec9d99750749211, Reformatted Input Tensor 0 to resnetv17_conv0_fwd + resnetv17_batchnorm0_fwd + resnetv17_relu0_fwd + resnetv17_pool0_fwd (Int8[-1,3,224,224]) -> resnetv17_pool0_fwd (Int8[-1,64:32,56,56])
Layer(CaskConvolution): resnetv17_stage1_conv0_fwd + resnetv17_stage1_batchnorm0_fwd + resnetv17_stage1_relu0_fwd, Tactic: 0x2640501019a61dc2, resnetv17_pool0_fwd (Int8[-1,64:32,56,56]) -> resnetv17_stage1_relu0_fwd (Int8[-1,64:32,56,56])
Layer(CaskConvolution): resnetv17_stage1_conv1_fwd + resnetv17_stage1_batchnorm1_fwd + resnetv17_stage1_relu1_fwd, Tactic: 0x9dafb2758560cc1d, resnetv17_stage1_relu0_fwd (Int8[-1,64:32,56,56]) -> resnetv17_stage1_relu1_fwd (Int8[-1,64:32,56,56])
Layer(CaskConvolution): resnetv17_stage1_conv2_fwd + resnetv17_stage1_batchnorm2_fwd, Tactic: 0xa433705d1adff009, resnetv17_stage1_relu1_fwd (Int8[-1,64:32,56,56]) -> resnetv17_stage1_batchnorm2_fwd (Int8[-1,256:32,56,56])
Layer(CaskConvolution): resnetv17_stage1_conv3_fwd + resnetv17_stage1_batchnorm3_fwd + resnetv17_stage1__plus0 + resnetv17_stage1_activation0, Tactic: 0x3ffcb62b1c6bb94f, resnetv17_pool0_fwd (Int8[-1,64:32,56,56]), resnetv17_stage1_batchnorm2_fwd (Int8[-1,256:32,56,56]) -> resnetv17_stage1_activation0 (Int8[-1,256:32,56,56])
Layer(CaskConvolution): resnetv17_stage1_conv4_fwd + resnetv17_stage1_batchnorm4_fwd + resnetv17_stage1_relu2_fwd, Tactic: 0x41bdb46e3c2617e5, resnetv17_stage1_activation0 (Int8[-1,256:32,56,56]) -> resnetv17_stage1_relu2_fwd (Int8[-1,64:32,56,56])
Layer(CaskConvolution): resnetv17_stage1_conv5_fwd + resnetv17_stage1_batchnorm5_fwd + resnetv17_stage1_relu3_fwd, Tactic: 0x9dafb2758560cc1d, resnetv17_stage1_relu2_fwd (Int8[-1,64:32,56,56]) -> resnetv17_stage1_relu3_fwd (Int8[-1,64:32,56,56])
Layer(CaskConvolution): resnetv17_stage1_conv6_fwd + resnetv17_stage1_batchnorm6_fwd + resnetv17_stage1__plus1 + resnetv17_stage1_activation1, Tactic: 0x3ffcb62b1c6bb94f, resnetv17_stage1_relu3_fwd (Int8[-1,64:32,56,56]), resnetv17_stage1_activation0 (Int8[-1,256:32,56,56]) -> resnetv17_stage1_activation1 (Int8[-1,256:32,56,56])
Layer(CaskConvolution): resnetv17_stage1_conv7_fwd + resnetv17_stage1_batchnorm7_fwd + resnetv17_stage1_relu4_fwd, Tactic: 0x41bdb46e3c2617e5, resnetv17_stage1_activation1 (Int8[-1,256:32,56,56]) -> resnetv17_stage1_relu4_fwd (Int8[-1,64:32,56,56])
...
on B200
[09/09/2025-11:28:18] [V] [TRT] Engine Layer Information:
Layer(Myelin): {ForeignNode[Quantize 0...resnetv17_dense0_fwd_castOut]}, Tactic: 0x0000000000000000, data (Float[-1,3,224,224]) -> resnetv17_dense0_fwd (Float[-1,1000])
it seems that the conversion doing less optimization on B200.
How to fix ?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the trtexec command and compare the verbose engine-layer output for resnet50-v1-12.onnx on H200 and B200, checking the TensorRT 25.08-py3 environment and shape profiles. Reproduce the conversion and benchmark both serialized engines; document whether the B200 engine's Myelin node changes performance and produce a minimal diagnostic reproducer or supported configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100