alibaba / alibaba/TinyNeuralNetwork

unnecessary float() variables cause quantization to fail.

Open
#281 7 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
879
Forks
134
PR merge metrics
No merged PRs in 30d

Description

I'm quantizing the [MI-GAN](https://github.com/Picsart-AI-Research/MI-GAN/) network that I have previously converted to tflite successfully with your help (thank-you!)

I'm basing my conversion off your https://github.com/alibaba/TinyNeuralNetwork/blob/main/examples/quantization/post.py script, and I've managed to work my way through some difficulties (especially convincing pytorch not to quietly convert floats to doubles, which then stop the conversion - a force double to float before quantizing switch might be a nice feature).

My current issue though seems an unnecessary one. I'm hitting:
"Creation of quantized tensor requires quantized dtype like torch.quint8" which appears to come from https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/quantized/TensorFactories.cpp line 115 or 128 or similar.

This is because in (Traceback):
`converter.convert()`
File "[longpath]/tinynn/converter/base.py", line 476, in convert
`self.init_jit_graph()`
File "[longpath]/tinynn/converter/base.py", line 228, in init_jit_graph
`script = torch.jit.trace(self.model, self.dummy_input)`
it reads a python script version of the model created earlier by `quantizer.quantize()` ([generator_q.py.txt](https://github.com/alibaba/TinyNeuralNetwork/files/14724618/generator_q.py.txt)
- .txt added to allow upload), which contains:
```
float_0_f = fake_quant_1.float() # line that causes failure
fake_quant_1 = None
```

[earlier: self.fake_quant_1 = torch.quantization.QuantStub() ]
`float_0_f` is then never used, so could just not exist, rather than cause a failure. There are dozens of these, and none of the float values are used.

Is there a way to stop these values being created? I can't immediately see where they come from to know if I can adjust the model or similar, but as they're unused, is there a way to have them automatically pruned?

A side question - is there a way to convert expected inputs from float to int as well? I have image input to the network that I have to convert from 0-256 to -1.0-1.0, so if there was a way to convert to sticking with integer, that would also be useful.

Contributor guide

Open the contributing guide

Research direction

Start with examples/quantization/post.py and reproduce the failure through converter.convert() in tinynn/converter/base.py, especially init_jit_graph(). Compare the generated generator_q.py.txt output around the unused .float() assignments. Done means unused float variables no longer cause quantization conversion to fail; the input float-to-int question is a separate requirement to clarify.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.