microsoft / microsoft/onnxruntime
MaxPool not getting quantized when preceded by Relu
Open
@zhanghuanrong is already working on this.
Since Oct 18, 2021.
quantization
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 179
Description
Describe the bug
MaxPool nodes are not getting quantized if a preceding Relu is not getting quantized.
Urgency
Development of a backend is blocked by this, so it would be great if someone could provide some insights as soon as possible.
System information
- ONNX Runtime installed from (source or binary): source
- ONNX Runtime version: 1.9.1
- Python version: 3.8
To Reproduce
- Run script below and inspect resulting quantized model
Expected behavior
Would expect a QuantizeLinear node before the Maxpool node in the quantized model.
import torch
import numpy as np
import onnx
import onnxruntime
from onnxruntime import quantization
IMAGE_SHAPE = (1, 1, 16, 16)
KERNEL_SHAPE = (1, 1, 2, 2)
class ToyModel(torch.nn.Module):
def __init__(self):
super().__init__()
self.conv = torch.nn.Conv2d(out_channels = KERNEL_SHAPE[0],
in_channels = KERNEL_SHAPE[1],
kernel_size = KERNEL_SHAPE[2:],
bias = False)
weight = torch.tensor(data = 127*np.ones(KERNEL_SHAPE).astype("float32"),
dtype = torch.float32)
self.conv.weight = torch.nn.Parameter(weight, requires_grad = False)
self.max_pool = torch.nn.MaxPool2d(kernel_size = (2,2))
def forward(self, input):
return self.max_pool(self.conv(input).relu())
class ToyDataProvider(onnxruntime.quantization.CalibrationDataReader):
def __init__(self, input_name):
self.data = ({ input_name: prefac*np.ones(IMAGE_SHAPE).astype("float32") } for prefac in [-127,+127])
def get_next(self):
try: return next(self.data)
except StopIteration: return None
def CreateToyModels(unquantized_path, quantized_path):
# Save toy model to onnx file
model = ToyModel()
torch.onnx.export(model, (torch.empty(IMAGE_SHAPE, dtype=torch.float32)), unquantized_path)
session = onnxruntime.InferenceSession(unquantized_path)
input_name = session.get_inputs()[0].name
# Quantize model with Relu using dummy data
onnxruntime.quantization.quantize_static(model_input = unquantized_path,
model_output = quantized_path,
calibration_data_reader = ToyDataProvider(input_name),
activation_type = onnxruntime.quantization.QuantType.QUInt8,
weight_type = onnxruntime.quantization.QuantType.QUInt8,
op_types_to_quantize = ["Conv", "MaxPool"],
extra_options = { "WeightSymmetric" : False,
"ActivationSymmetric" : False })
if __name__ == "__main__":
# Create quantized and unquantized models
CreateToyModels(unquantized_path = "model.onnx", quantized_path = "quantized-model.onnx")
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.