onnxruntime 1.17.0: transformers benchmarking failing for int8 quantized inference.

### Describe the issue

Onnxruntime transformers benchmarking is failing for int8 quantized inference. the same is working fine with onnxruntime 1.16.3. I added the error details below.
I found the below commit ( commit c8399a81fed9c114c43daf2103fee48d6b02bdd7: adding support for float16 weights quantization) is causing the break, but the commit already says the feature will not work with onnx 1.15. So, I tried with onnx-weekly (onnx-weekly 1.16.0.dev20240130) but the issue still exist.
My question is, which onnx version is required to make this test work again?

**Error:**
```
Quantization parameters for tensor:"/embeddings/LayerNorm/Add_1_output_0" not specified
Quantization parameters for tensor:"/encoder/layer.0/attention/self/Reshape_3_output_0" not specified
Exception
Traceback (most recent call last):
File "/home/ubuntu/sn_onnxruntime/onnxruntime/onnxruntime/python/tools/transformers/benchmark.py", line 902, in main
results += run_onnxruntime(
File "/home/ubuntu/sn_onnxruntime/onnxruntime/onnxruntime/python/tools/transformers/benchmark.py", line 160, in run_onnxruntime
) = export_onnx_model_from_pt(
File "/home/ubuntu/sn_onnxruntime/onnxruntime/onnxruntime/python/tools/transformers/onnx_exporter.py", line 550, in export_onnx_model_from_pt
onnx_model_file, is_valid_onnx_model, vocab_size = validate_and_optimize_onnx(
File "/home/ubuntu/sn_onnxruntime/onnxruntime/onnxruntime/python/tools/transformers/onnx_exporter.py", line 439, in validate_and_optimize_onnx
QuantizeHelper.quantize_onnx_model(onnx_model_path, onnx_model_path, use_external_data_format)
File "/home/ubuntu/sn_onnxruntime/onnxruntime/onnxruntime/python/tools/transformers/quantize_helper.py", line 68, in quantize_onnx_model
quantize_dynamic(
File "/home/ubuntu/.local/lib/python3.10/site-packages/onnxruntime/quantization/quantize.py", line 642, in quantize_dynamic
quantizer.quantize_model()
File "/home/ubuntu/.local/lib/python3.10/site-packages/onnxruntime/quantization/onnx_quantizer.py", line 403, in quantize_model
op_quantizer.quantize()
File "/home/ubuntu/.local/lib/python3.10/site-packages/onnxruntime/quantization/operators/matmul.py", line 78, in quantize
otype = self.quantizer.get_tensor_type(node.output[0], mandatory=True)
File "/home/ubuntu/.local/lib/python3.10/site-packages/onnxruntime/quantization/onnx_quantizer.py", line 461, in get_tensor_type
raise RuntimeError(f"Unable to find data type for weight_name={tensor_name!r}")
RuntimeError: Unable to find data type for weight_name='/encoder/layer.0/attention/output/dense/MatMul_output_0'
```

**Commit that introduced this**:
```
commit c8399a81fed9c114c43daf2103fee48d6b02bdd7
Author: Xavier Dupré
Date: Fri Jan 12 17:54:55 2024 +0100

Quantization tool: support float 8 with MatMul, support float 16 weights (#18043)

### Description

Whenever a node QuantizeLinear or DequantizeLinear, the type of the
weights before being quantize must be known to create the scale with the
expected type. Another option would be to add many operator CastLike but
that would push the burden to onnxruntime optimizer.

The PR tries to avoid changing the signature. To do so, it modified the
scale computation to use a numpy array to store the result and not a
python float. The numpy array must be of the same type than the weights
to quantize.

The PR adds many `assert` to check the type of the scale is not a python
type or a float64. This was added to make sure all the code follows the
same logic. These lines were kept for the first review.

DequantizeLinear, QuantizeLinear cannot be tested with onnx==1.15. PR
https://github.com/onnx/onnx/pull/5709 is missing to fix shape
inference. PR https://github.com/onnx/onnx/pull/5473) is missing to
support QLinearMatMul with float 16. That explains why some tests are
disabled with float 16.

### Motivation and Context

The current quantization tool assumes every weight is float 32. For
large models such as LLAMA, it is usually float 16. The quantization
needs to quantize such weights.
```

### To reproduce

```
pip install onnxruntime==1.17.0
pip install onnx or pip install onnx-weekly

git clone https://github.com/microsoft/onnxruntime.git
cd onnxruntime
git submodule sync
git submodule update --init --recursive
cd onnxruntime/python/tools/transformers
python3 benchmark.py -p int8 -m bert-base-cased
```

### Urgency

it is actually blocking new features development and testing, because the tests are no longer working.

### Platform

Linux

### OS Version

Ubuntu 22.04

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.17.0

### ONNX Runtime API

Python

### Architecture

ARM64

### Execution Provider

Default CPU

### Execution Provider Library Version

_No response_