How TRT Fuses Q/DQ Operators with the Kernels

Question

I would like to understand more on how TRT under the hood converts Q/DQ models to use actual INT8 kernels. Specifically, how are the scale factors being propagated. Specifically I wonder the two cases described here (I don't need to understand how Q/DQ got propagated for now)

When I used TRT modelopt to quantize a Resnet18 from torch, I noticed that the native (Fake Quantization with Q/DQ) generates different outputs comparing to the TRT compiled model. I would like to develop a tooling that can figure out which layer causes this regression in the native framework (Pytorch).

The current idea I have is to get the output of the Q node for both input and weight, perform the INT8 operation, then apply the DQ nodes to rescale back the output. I do this because algebraically:

Conv(DQ_x(Q_x(X), DQ_w(Q_w(W)) 
= Conv(1/s_x * clip_round(s_x * X), 1/s_w * clip_round(s_x * W))
= Conv(clip_round(s_x * X), clip_round(s_x * W)) / s_x / s_w
= DQ_output(Conv(Q_x(X), Q_w(W))

I noticed that there is a regression with this approach already even if I am using FP32 kernel. I think the regression comes from floating point ops are not really transitive. However, for TRT, when only compiling one layer, there is no regression. Therefore, I wonder if you can shed some lights on how this computation is performed under the hood for TRT, so that I can simulate this Layer-wise diffing in the native framework.

Thank you in advance,

The issue names no repository files, tests, or entry points to start from. Begin by reproducing the ResNet18 TensorRT Model Optimizer Q/DQ comparison and determine how a layer-wise diff should account for scale propagation and fused INT8 kernels; done would require an agreed explanation or scoped tooling plan.