Explicit INT8 Quantization Fails to Fuse Concat-Conv Block Compared to Implicit Mode

Description

I am currently trying to quantize my UNet-based pytorch model to INT8. I end up with a slightly different —and slower— engine graph compared to the one generated by implicit quantization.

More specifically, I’m struggling to properly quantize the Concat-Conv section of the network. I’ve tried all suggested Q/DQ placements from various issues (and more), but TensorRT never fully fuses the operators in that section. Since I can’t seem to find an official Q/DQ placement guide in the documentation anymore, I’m reaching out for your advice.

PS: I think it would be very helpful to include a Q/DQ placement guide in the official documentation again.

Below, I’ll show a minimal example of the ONNX and TensorRT engine graphs I experimented with.

This is the fp32 ONNX model and the TRT engine generated by implicit quantization:

When i use the basic configuration of the pytorch_quantization toolkit, which means i don't quantize concat (as proposed in this issue https://github.com/NVIDIA/TensorRT/issues/3861), i get these results:

The second thing tried was adding Q/DQ nodes to both inputs of concat (as proposed in this issue https://github.com/NVIDIA/TensorRT/issues/1144), this is the result:

Then i only quantized the residual path, which is my best try so far, however there are still unfused scale/pointwise operators:

If you have any idea how to correctly place the Q/DQ nodes in this scenario to match implicit quantization behavior, I’d really appreciate your insights. Also, if there’s a hidden or updated Q/DQ placement guide somewhere, I’d be very grateful if you could point me to it.
Thank you so much for your help and time!

@ttyio tagging you since you’ve provided valuable input on many INT8 quantization issues :)

Environment

TensorRT Version: 10.8.0.43

NVIDIA GPU: GeForce RTX 4090

NVIDIA Driver Version: 550.120

CUDA Version: 12.8.0.38

CUDNN Version: 9.7.1.26

The issue provides no repository file or test entry point. Start by reproducing the minimal UNet Concat-Conv graphs with TensorRT 10.8 and comparing implicit versus explicit INT8 engine fusion. A useful resolution would document the supported Q/DQ placement or identify the placement needed to eliminate the remaining scale and pointwise operators.