[Performance] MatMulNBits Performance

### Describe the issue

Why do `MatMulNBits` operators with quant types `int4`/`uint4` (both `f32` and `f16` as dtypes) perform at least 10x slower than `MatMulIntegerToFloat`/`DynamicQuantizeMatMul` in the dynamic quantization process (`int8/uint8`) on CPUs across platforms?

- Linux+Intel
- Windows+Intel
- Android+Qualcomm+Arm64-v8.6
- Mac+M3

The test models range from small LLMs (0.5B to 3B parameters), including Llama, Phi, Qwen, and Gemma, all showing the same performance results. Any insights?

### To reproduce

INT4:
```
from onnxruntime.quantization import (
matmul_4bits_quantizer,
quant_utils,
quantize
)
from pathlib import Path

model_fp32_path="path/to/orignal/model.onnx"
model_int4_path="path/to/save/quantized/model.onnx"

quant_config = matmul_4bits_quantizer.DefaultWeightOnlyQuantConfig(
block_size=256, # 2's exponential and >= 16
is_symmetric=True, # if true, quantize to Int4. otherwsie, quantize to uint4.
accuracy_level=4, # used by MatMulNbits, see https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md#attributes-35
quant_format=quant_utils.QuantFormat.QOperator,
op_types_to_quantize=("MatMul","Gather"), # specify which op types to quantize
quant_axes=(("MatMul", 0), ("Gather", 1),) # specify which axis to quantize for an op type.

model = quant_utils.load_model_with_shape_infer(Path(model_fp32_path))
quant = matmul_4bits_quantizer.MatMul4BitsQuantizer(
model,
nodes_to_exclude=None, # specify a list of nodes to exclude from quantizaiton
nodes_to_include=None, # specify a list of nodes to force include from quantization
algo_config=quant_config,)
quant.process()
quant.model.save_model_to_file(
model_int4_path,
True) # save data to external file
```

INT8:
```
quantize_dynamic(
model_input=model_path,
model_output=quanted_model_path,
per_channel=True,
reduce_range=False,
weight_type=QuantType.QUInt8,
extra_options={'ActivationSymmetric': True,
'WeightSymmetric': True,
'EnableSubgraph': True,
'ForceQuantizeNoInputCheck': False,
'MatMulConstBOnly': True
},
nodes_to_exclude=None,
use_external_data_format=False
)
```

### Urgency

None

### Platform

Linux

### OS Version

22.04

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.20.1

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

Default CPU

### Execution Provider Library Version

1.20.1

### Model File

_No response_

### Is this a quantized model?

Yes