tensorflow / tensorflow/model-optimization

[Feature request or potential bug] Override of default default_8bit_quantize_layout_transform

Open
#445 10 comments 0 reactions 2 assignees View on GitHub

@Xhark is already working on this.

Since Apr 18, 2021.

feature request technique:qat
Dominant language
Python
Stars
1.6k
Forks
349
Avg merge
3d 2h
Merged PRs (30d)
1

Description

In file

model-optimization/tensorflow_model_optimization/python/core/quantization/keras/quantize.py

at line 407 (function quantize_apply(model)) the model is transformed using the default_8bit_quantize_layout_transform that is necessary for the edge tpu. However, this can interfere if the quantization is custom since there is no obvious way to avoid it and the only way to know what is happening is to actually open the source code.

Note that this may virtually be a serious bug invalidating all the work done using this API: all quantized model, independently from their annotation, get transformed using the default transform defined in

tensorflow_model_optimization\python\core\quantization\keras\default_8bit\default_8bit_quantize_layout_transform.py

This involves unwanted changes on several layers, the most critical being the input layer (that gets transformed with an undocumented MovingAverageQuantizer) and the batch norm (that gets fused with the conv layers). Virtually every research paper written using this API has to be withdrawn or amended.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.