[Performance] Share weights between sessions to accelerate inference

### Describe the issue

I'm trying to share base model weights between ort inference sessions to accelerate inference for adapter models. Based on the API documentation and previous conversation in https://github.com/microsoft/onnxruntime/issues/15301, I:
1. manually export base model and adapter weights
2. load base model weights into tensor
3. add them to the session_options object using `add_initializer` API
4. create an inference session using the session_options object in step 3
5. for a different adapter weights, repeat step 3-4

I expect the following calls to create an inference session to be much faster than the 1st call since we reuse the base model weights, but I'm getting the same latency for creating an inference session for all the following calls. Did I miss anything here?

### To reproduce

Below is the sample python code snippet I used for testing:

```python
def create_inference_session(base_model, adapter_tensors, base_model_tensors):
# Add base model weights to session options
start = time.time()
opts = ort.SessionOptions()
for name, data in base_model_tensors.items():
opts.add_initializer(name, data)
end = time.time()
print(f'Base model adding time: {end - start}')

start = time.time()
for tensor in adapter_tensors:
opts.add_initializer(tensor[0], tensor[1])
end = time.time()
print(f'Adapter adding time: {end - start}')

# create a new inference session
start = time.time()
session = ort.InferenceSession(base_model.SerializeToString(), sess_options=opts, providers=['CPUExecutionProvider'])
end = time.time()
print(f'Session creation time: {end - start}')
```

### Urgency

_No response_

### Platform

Linux

### OS Version

Ubuntu 22.04.4 LTS

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.15.1

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

Default CPU

### Execution Provider Library Version

_No response_

### Model File

_No response_

### Is this a quantized model?

No