alibaba / alibaba/TinyNeuralNetwork
unnecessary float() variables cause quantization to fail.
- Dominant language
- Python
- Stars
- 879
- Forks
- 134
- PR merge metrics
- No merged PRs in 30d
Description
I'm quantizing the [MI-GAN](https://github.com/Picsart-AI-Research/MI-GAN/) network that I have previously converted to tflite successfully with your help (thank-you!)
I'm basing my conversion off your https://github.com/alibaba/TinyNeuralNetwork/blob/main/examples/quantization/post.py script, and I've managed to work my way through some difficulties (especially convincing pytorch not to quietly convert floats to doubles, which then stop the conversion - a force double to float before quantizing switch might be a nice feature).
My current issue though seems an unnecessary one. I'm hitting:
"Creation of quantized tensor requires quantized dtype like torch.quint8" which appears to come from https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/quantized/TensorFactories.cpp line 115 or 128 or similar.
This is because in (Traceback):
`converter.convert()`
File "[longpath]/tinynn/converter/base.py", line 476, in convert
`self.init_jit_graph()`
File "[longpath]/tinynn/converter/base.py", line 228, in init_jit_graph
`script = torch.jit.trace(self.model, self.dummy_input)`
it reads a python script version of the model created earlier by `quantizer.quantize()` ([generator_q.py.txt](https://github.com/alibaba/TinyNeuralNetwork/files/14724618/generator_q.py.txt)
- .txt added to allow upload), which contains:
```
float_0_f = fake_quant_1.float() # line that causes failure
fake_quant_1 = None
```
[earlier: self.fake_quant_1 = torch.quantization.QuantStub() ]
`float_0_f` is then never used, so could just not exist, rather than cause a failure. There are dozens of these, and none of the float values are used.
Is there a way to stop these values being created? I can't immediately see where they come from to know if I can adjust the model or similar, but as they're unused, is there a way to have them automatically pruned?
A side question - is there a way to convert expected inputs from float to int as well? I have image input to the network that I have to convert from 0-256 to -1.0-1.0, so if there was a way to convert to sticking with integer, that would also be useful.
Contributor guide
Research direction
Start with examples/quantization/post.py and reproduce the failure through converter.convert() in tinynn/converter/base.py, especially init_jit_graph(). Compare the generated generator_q.py.txt output around the unused .float() assignments. Done means unused float variables no longer cause quantization conversion to fail; the input float-to-int question is a separate requirement to clarify.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100