microsoft / microsoft/onnxruntime

Static quantization crashes with "TypeError: iteration over a 0-d array"

Open
#19,529 2 comments 0 reactions 1 assignee Claimed by @xadupre View on GitHub
quantization
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

During static quantization with percentile calibration, the following crash happens with onnxruntime-gpu 1.17.0:

Collecting tensor data and making histogram ...
Traceback (most recent call last):
File ".\new_onnx_quantize_static.py", line 21, in
oq.quantize_static(preprocessed_path, quantized_path, Mock(),
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\quantize.py", line 496, in quantize_static
calibrator.collect_data(calibration_data_reader)
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\calibrate.py", line 546, in collect_data
self.collector.collect(clean_merged_dict)
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\calibrate.py", line 724, in collect
return self.collect_absolute_value(name_to_arr)
File "D:\Tools\Venv\NewestOnnx\lib\site-packages\onnxruntime\quantization\calibrate.py", line 739, in collect_absolute_value
dtypes = set(a.dtype for a in arr)
TypeError: iteration over a 0-d array

Older onnxruntime versions did not have this issue.

### To reproduce

Model is: [minimal_model.zip](https://github.com/microsoft/onnxruntime/files/14297194/minimal_model.zip). Code to reproduce:

import onnxruntime.quantization as oq
import numpy as np

class Mock:
def __init__(self):
self.i = 0

def get_next(self):
if self.i > 10:
return None
self.i += 1
return {"input": np.random.randint(0, 255, size=(1, 3, 32, 32), dtype=np.uint8)}

if __name__ == "__main__":
onnx_path = "minimal_model.onnx"
preprocessed_path = onnx_path[:-5] + "_preprocessed.onnx"
quantized_path = onnx_path[:-5] + "_quantized.onnx"
oq.quant_pre_process(onnx_path, preprocessed_path, skip_symbolic_shape=True)
oq.quantize_static(preprocessed_path, quantized_path, Mock(),
calibrate_method=oq.CalibrationMethod.Percentile,
op_types_to_quantize=["Conv", "Mul", "Gemm"])

### Urgency

Urgent, since it worked with previous versions of onnxruntime-gpu.

### Platform

Windows

### OS Version

10

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.17.0

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

CUDA

### Execution Provider Library Version

CUDA 11.6

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.