NVIDIA / NVIDIA/TensorRT

FP16 model compilation failure with TensorRT 11.2.1.2 on 4090 RTX

Open
#4,838 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Module:ONNX
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

Some models with FP16 precision fail to compile from ONNX format with error like this:

No low-precision conv kernel available for this strongly-typed Conv/ConvTranspose. Consider keeping Conv/ConvTranspose and Q/DQ ops in FP32 (e.g. via modelopt.onnx.autocast.convert_to_f16 with op_block_list=['Conv','ConvTranspose','QuantizeLinear','DequantizeLinear']). Failing node: node_223_Conv
[08/25/2026-19:48:50] [E] Error[10]: IBuilder::buildSerializedNetworkToStream: Error Code 10: Internal Error (Could not find any implementation for node node_223_Conv. In computeCosts at /_src/optimizer/common/tactic/optimizer.cpp:4211)

TensorRT 10.16.1.11 compiles these models successfully. Also, FP32 works normally in TensorRT 11.2.1.2.

I reproduced the error with the simplest ONNX sample consisted of Input->Cast(FP16)->Conv(Grouped)->Cast(FP32)->Output (attachead
model.onnx.zip)
Probably problem related to group Conv optimizations.

Full failure log attached 11.2.1.2.log
Also 10.16.1.11 successful conversion log attached 10.16.1.11.log
Also attached engine compiled with 10.16.1.11 engine_10.16.1.11.trt.zip

Environment

TensorRT Version: 11.2.1.2

NVIDIA GPU: 4090 RTX

NVIDIA Driver Version: 610.57.04

CUDA Version: 13.3

Operating System: Ubuntu 22.04

Steps To Reproduce

Commands or scripts: trtexec --onnx=model.onnx --saveEngine=engine.trt

Have you tried the latest release?: Yes

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):Yes

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the attached model.onnx.zip and reproduce the failure using trtexec --onnx=model.onnx --saveEngine=engine.trt under TensorRT 11.2.1.2, then compare against the successful 10.16.1.11 log and engine. Investigate the grouped Conv FP16 compilation path; done means the supplied model compiles successfully on TensorRT 11.2.1.2 without requiring the FP32 workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.