tensorflow / tensorflow/model-optimization
[Feature request or potential bug] Override of default default_8bit_quantize_layout_transform
@Xhark is already working on this.
Since Apr 18, 2021.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 349
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 1
Description
In file
model-optimization/tensorflow_model_optimization/python/core/quantization/keras/quantize.py
at line 407 (function quantize_apply(model)) the model is transformed using the default_8bit_quantize_layout_transform that is necessary for the edge tpu. However, this can interfere if the quantization is custom since there is no obvious way to avoid it and the only way to know what is happening is to actually open the source code.
Note that this may virtually be a serious bug invalidating all the work done using this API: all quantized model, independently from their annotation, get transformed using the default transform defined in
tensorflow_model_optimization\python\core\quantization\keras\default_8bit\default_8bit_quantize_layout_transform.py
This involves unwanted changes on several layers, the most critical being the input layer (that gets transformed with an undocumented MovingAverageQuantizer) and the batch norm (that gets fused with the conv layers). Virtually every research paper written using this API has to be withdrawn or amended.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.