the latency of the fp16 onnx model is longger than that for the fp32 hg pytorch model

Describe the bug
Hello，I tired to export the fp32 hugging face bart model to fp 16 onnx model deployed on NVIDIA A100 with onnxruntime. But the latency for the fp 16 onnx model is almost twice( 75ms ) that for fp32 hugging face pytorch model (52ms).

The process is listed below:

1. Export hugging face pytorch bart model to onnx format using the onnx config tool provided by hg:
python -m transformers.onnx --model=facebook/bart-large bart-large.onnx

2. Optimizing the onnx model and converting it to the fp16 model:

python -m onnxruntime.transformers.optimizer --input bart-large.onnx --output optbart-large.onnx --float16 --opt_level 99 --model_type bert--use_gpu

3. run the model with onnxruntime:

inputs = torch.randint(low=1, high=15000, size=(2, 1024))

input_ort = {
'input_ids' : inputs.numpy(),
"attention_mask": np.ones_like(inputs)
}
latency1 = []

def evaluate_onnx(model_path):
  execution_providers = ['CUDAExecutionProvider', 'CPUExecutionProvider']
  sess_options = onnxruntime.SessionOptions()
  sess_options.execution_mode = onnxruntime.ExecutionMode.ORT_SEQUENTIAL
  sess_options.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_ENABLE_ALL
  session = onnxruntime.InferenceSession(model_path, sess_options, providers=execution_providers)
  session.disable_fallback()
  assert 'CUDAExecutionProvider' in session.get_providers()
  for i in range(50):
    start = time.time()
   session.run(None, input_ort)
   latency1.append(time.time() - start)
  print(latency1)

Urgency
If there are particular important use cases blocked by this or strict project-related timelines, please share more information and dates. If there are no hard deadlines, please specify none.
We are blocked by the latency of bart model.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux
• ONNX Runtime installed from (source or binary): pip install -U onnxruntime-gpu
• ONNX Runtime version: 1.8.1
• Python version: 3.6
• Visual Studio version (if applicable):
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version: CUDA11.1/CUDNN 8.05
• GPU model and memory: A100 32GB

To Reproduce

• Describe steps/code to reproduce the behavior.
• Attach the ONNX model to the issue (where applicable) to expedite investigation.

Expected behavior
the latency of the fp16 onnx model is longger than that for the fp32 hg pytorch model

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here. If the issue is about a particular model, please share the model details as well to facilitate debugging.