ONNX Runtime performing worse than PyTorch on BERT CPU Inference

@tianleiwu is already working on this.

Since Jul 29, 2020.

**Describe the bug**

I run this [notebook](https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/transformers/notebooks/PyTorch_Bert-Squad_OnnxRuntime_CPU.ipynb) on my machine and I can not replicate the performance improvement results.

**System information**
Macbook Pro:
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): 2.6 GHz 6-Core Intel Core i7
EC2 Instance:
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
- ONNX Runtime installed from (source or binary): binary
- ONNX Runtime version: 1.3.0
- Python version: 3.6.8

**To Reproduce**
- Run this notebook https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/transformers/notebooks/PyTorch_Bert-Squad_OnnxRuntime_CPU.ipynb

**Expected behavior**
I expected results similar to the ones cached in the notebook:

* PyTorch : `198.99 ms`
* ONNX : `176.96 ms`
* ONNX best model after grid search: `84.73 ms` (with `OMP_NUM_THREAD = 12`, `intra_num_threads = 1`)
* ONNX worse model: `306.75 ms` (with `OMP_NUM_THREAD = 1`, `intra_num_threads = 1`)

But on my Mac I got:
* PyTorch : `106.82 ms`
* ONNX : `265.84 ms`
* ONNX best model after grid search: `234.48 ms` (with `OMP_NUM_THREAD = 12`, `intra_num_threads = 1`)
* ONNX worse model: `273.45 ms` (with `OMP_NUM_THREAD = 1`, `intra_num_threads = 1`)

And on EC2 :
* PyTorch : `137.08 ms`
* ONNX : `189.88 ms`
* ONNX best model after grid search: `175.70 ms` (with `OMP_NUM_THREAD = 4`, `intra_num_threads = 1`)
* ONNX worse model: `267.25 ms` (with `OMP_NUM_THREAD = 1`, `intra_num_threads = 1`)

Thanks in advance!

cc: @tianleiwu