Failed INT8 quantization.

Dear Developers,

I am very new to Tensorrt and quantization. Previously I only use the basic example of Tensorrt to generate engines in FP16 because I thought INT8 will compromise accuracy significantly. Until recently I realized there are methods to improve the accuracy of a quantized model, so I started learning how to do it in Tensorrt.

I can successfully convert my segmentation model with ResNet34 as backbone to both FP16 and INT8 without any issue. But when I switch the backbone to efficientnet_b2, the model can still be converted to FP16, but it fails with the following error message when I'm trying to quantize it to INT8. I tried to increase the workspace as some other threads suggested, but it doesn't help.

onnx files of the models mentioned:
EfficientNet-backbone model onnx
ResNet34-backbone model onnx

[TensorRT] VERBOSE: --------------- Timing Runner: Conv_22 + PWN(PWN(Sigmoid_23), Mul_24) (CudaDepthwiseConvolution) [TensorRT] VERBOSE: CudaDepthwiseConvolution has no valid tactics for this config, skipping [TensorRT] VERBOSE: --------------- Timing Runner: Conv_22 + PWN(PWN(Sigmoid_23), Mul_24) (CudnnConvolution) [TensorRT] VERBOSE: CudnnConvolution has no valid tactics for this config, skipping [TensorRT] VERBOSE: --------------- Timing Runner: Conv_22 + PWN(PWN(Sigmoid_23), Mul_24) (CaskConvolution) [TensorRT] VERBOSE: CaskConvolution has no valid tactics for this config, skipping [TensorRT] VERBOSE: *************** Autotuning format combination: Half(65536,1:16,256,1) -> Half(65536,1:16,256,1) *************** [TensorRT] VERBOSE: --------------- Timing Runner: Conv_22 + PWN(PWN(Sigmoid_23), Mul_24) (CudnnConvolution) [TensorRT] VERBOSE: CudnnConvolution has no valid tactics for this config, skipping [TensorRT] VERBOSE: --------------- Timing Runner: Conv_22 + PWN(PWN(Sigmoid_23), Mul_24) (CaskConvolution) [TensorRT] VERBOSE: CaskConvolution has no valid tactics for this config, skipping [TensorRT] VERBOSE: *************** Autotuning format combination: Half(65536,65536:32,256,1) -> Half(65536,65536:32,256,1) *************** [TensorRT] VERBOSE: --------------- Timing Runner: Conv_22 + PWN(PWN(Sigmoid_23), Mul_24) (CudnnConvolution) [TensorRT] VERBOSE: CudnnConvolution has no valid tactics for this config, skipping [TensorRT] VERBOSE: --------------- Timing Runner: Conv_22 + PWN(PWN(Sigmoid_23), Mul_24) (CaskConvolution) [TensorRT] VERBOSE: CaskConvolution has no valid tactics for this config, skipping [TensorRT] VERBOSE: Deleting timing cache: 881 entries, 1138 hits [TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 4763, GPU 9038 (MiB) [TensorRT] ERROR: 10: [optimizer.cpp::computeCosts::1855] Error Code 10: Internal Error (Could not find any implementation for node Conv_22 + PWN(PWN(Sigmoid_23), Mul_24).) [TensorRT] ERROR: 2: [builder.cpp::buildSerializedNetwork::417] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed.) Completed creating Engine Traceback (most recent call last): File "basnet_test.py", line 231, in <module> main(batch_size=bs, image_size=(256, 256), proc_size=(256, 256), input_size=(720, 1280), model_name='effnet_b2' File "basnet_test.py", line 175, in main f.write(serialized_engine) TypeError: a bytes-like object is required, not 'NoneType'

The code I used to generate the engine is almost the same as the official example:

https://github.com/deephog/code/blob/61f3d9f445296ebd7fe2593b7f6ba034017471cc/question#L1-L26

I tried to have FP16 flag up, to give the engine more flexibility, but the error still persisted.

Please share any of your thoughts or suggestions. Much appreciated!

Reproduce the INT8 build using basnet_test.py, the linked EfficientNet-B2 and ResNet34 ONNX files, and the referenced official example. Start with the verbose log around Conv_22 and compare the FP16 and INT8 paths. Done means a reproducible diagnosis or confirmed resolution for the missing implementation, with engine creation no longer returning None.