microsoft / microsoft/onnxruntime
Static quantized resnet is slower than the raw one
@yufenglee is already working on this.
Since Sep 29, 2020.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
**Describe the bug**
Static quantized resnet is slower than the raw one.
Compared the static quantized Resnet model and the raw one from the E2E example code.
here is some benchmark from my computer
resnet50_v1.onnx / 3.03125
resnet50_v1.quant.onnx / 6.484375
here are the code piece
```
def perf_test(onnx_path, num=10):
sess = onnxruntime.InferenceSession(onnx_path)
name = sess.get_inputs()[0].name
data = np.random.random((1, 224, 224, 3)).astype(np.float32)
ort_in = {name: data}
# warm up
sess.run(None, ort_in)
latency = 0
for i in range(num):
start_32 = time.process_time()
ort_out = sess.run(None, ort_in)
latency = latency + time.process_time() - start_32
return latency
def latency_benchmark(num=10):
models = [
"resnet50_v1.onnx",
"resnet50_v1.quant.onnx",
]
latency_dict = {model: perf_test(model, num) for model in models}
for m, l in latency_dict.items():
print(f"{m} / {l}")
```
**Urgency**
Quantization performance
**System information**
Windows server
onnx==1.7.0
onnxruntime==1.4.0
Python version: 3.6
**To Reproduce**
See bug description
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.