INT8 quantization for HiFi-GAN vocoder -- performance issue

Hello!

My goal is to run trt engine for Hi-Fi GAN on H100 as efficiently as possible with small or none drop in quality. I have decided that the simplest path to achieve this would be as follows:

1. To quantize the model to int8 via TensorRT-Model-Optimizer (modelopt) library.
2. To finetune the quantized model (as per instructions).
3. To convert the quantized model into trt using the --stronglyTyped option of trtexec, so that no unpredictable additional quantization is done by trtexec.

If I skip the first two steps and quantize the model using --int8 flag of trtexec, the model runs 2 times faster. In contrast, after quantization with modelopt library, there is no speedup at all for the resulting trt engine. Please see the detailed times in the corresponding issue at the modelopt repo.

Could you please give me some recommendations regarding quantization of the model?

1. Is there any other way for finetuning the int8 version of the model before deploying it as a trt-engine?
2. What would be the proper QDQ placement for the model for achieving speedup after quantization?
3. If I convert the model to trtexec right away, the difference between --fp16 and --int8 flags is small. Shouldn't in theory --fp16 give the 2x speedup, while --int8 -- 4x?

Here is the link to the unquantized onnx model: https://drive.google.com/file/d/1qVotIH-0K73rXUlZ6yoPgIvOjlEOnvGY/view?usp=sharing

Start with the linked TensorRT-Model-Optimizer issue and the timing details referenced there. Reproduce the HiFi-GAN ONNX engine builds on H100 using the stated modelopt, --stronglyTyped, --int8, and --fp16 paths, then compare performance and quality. Done means documenting a supported quantization path and explaining the QDQ placement or missing speedup.