MxNet Converted Arcface Model Slow Compared to Provided Arcface Model

I have a arcface/resnet100 model that I've trained using InsightFace's MxNet training. For inference, I have converted the model to ONNX with the help of https://github.com/linghu8812/tensorrt_inference/blob/master/project/arcface/export_onnx.py.

The inference results are correct on my converted model, however speed of the model is extremely slow. For reference, I compared with the arcface model provided in this repository (arcfaceresnet100-8.onnx). Inference when running my model takes ~7 seconds, whereas the other model takes < 1 second.

When comparing the two models in Netron, all of the nodes, attributes, input/output shapes are the same (weights are different, obviously), however when I run the onnx profiler on the two models, there are a few differences. I've attached the profile logs for both models.

profile_arcfaceresnet100-8.txt
profile_model-opt.txt

There are a few differences in the two logs. Mainly (mine vs. arcfaceresnet100-8):

• ReorderInput/ReorderOutput operations: 99 vs. 51
• Conv operations: 103 vs. 152
• BatchNormalization operations: 51 vs. 2
• Avg. PRelu time: 995.8μs vs. 129.1μs
• Avg. Conv time: 12079.6μs vs. 411.9μs

I am not sure what the differences are when converting. It is critical that I get my converted model to run with a similar performance as the arcfaceresnet100-8 model. I've tried running my model through simplifiers/optimizers/etc., but with no improvement.

Here are my environment details:

OS: Linux Ubuntu Server 20.04
Python: 3.8

MxNet version: 1.9.1
ONNXRuntime: 1.14.0
ONNX 1.13.0
ONNX IR Version: 8
ONNX Opset Version: 18

If anyone could provide insight as to why my model performs slower or why there are differences in execution, that would be extremely helpful.

Start with the referenced project/arcface/export_onnx.py conversion script and compare its output with arcfaceresnet100-8.onnx. Run the ONNX Runtime benchmarks and inspect profile_arcfaceresnet100-8.txt and profile_model-opt.txt, focusing on the reported convolution, batch-normalization, and reorder differences. Done means identifying the conversion or execution difference and documenting a reproducible path to comparable inference performance.