microsoft / microsoft/onnxruntime
Static quantization crashes with "TypeError: iteration over a 0-d array"
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
During static quantization with percentile calibration, the following crash happens with onnxruntime-gpu 1.17.0:
Collecting tensor data and making histogram ...
Traceback (most recent call last):
File ".\new_onnx_quantize_static.py", line 21, in
oq.quantize_static(preprocessed_path, quantized_path, Mock(),
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\quantize.py", line 496, in quantize_static
calibrator.collect_data(calibration_data_reader)
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\calibrate.py", line 546, in collect_data
self.collector.collect(clean_merged_dict)
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\calibrate.py", line 724, in collect
return self.collect_absolute_value(name_to_arr)
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\calibrate.py", line 739, in collect_absolute_value
dtypes = set(a.dtype for a in arr)
TypeError: iteration over a 0-d array
Older onnxruntime versions did not have this issue.
### To reproduce
Model is: [minimal_model.zip](https://github.com/microsoft/onnxruntime/files/14297194/minimal_model.zip). Code to reproduce:
import onnxruntime.quantization as oq
import numpy as np
class Mock:
def __init__(self):
self.i = 0
def get_next(self):
if self.i > 10:
return None
self.i += 1
return {"input": np.random.randint(0, 255, size=(1, 3, 32, 32), dtype=np.uint8)}
if __name__ == "__main__":
onnx_path = "minimal_model.onnx"
preprocessed_path = onnx_path[:-5] + "_preprocessed.onnx"
quantized_path = onnx_path[:-5] + "_quantized.onnx"
oq.quant_pre_process(onnx_path, preprocessed_path, skip_symbolic_shape=True)
oq.quantize_static(preprocessed_path, quantized_path, Mock(),
calibrate_method=oq.CalibrationMethod.Percentile,
op_types_to_quantize=["Conv", "Mul", "Gemm"])
### Urgency
Urgent, since it worked with previous versions of onnxruntime-gpu.
### Platform
Windows
### OS Version
10
### ONNX Runtime Installation
Released Package
### ONNX Runtime Version or Commit ID
1.17.0
### ONNX Runtime API
Python
### Architecture
X64
### Execution Provider
CUDA
### Execution Provider Library Version
CUDA 11.6
Contributor guide
Assessment
This issue has not been assessed yet.