tensorflow / tensorflow/model-optimization
Weight in fully connected layers don't follow tensorflow quantization spec (zero-point!=0)
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 349
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 1
Description
1. System information
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04.5 LTS
- TensorFlow installation (pip package or built from source): pip
- TensorFlow library (version, if pip package or github SHA, if built from source): tensorflow2.5.0
2. Code
Provide code to help us reproduce your issues using one of the following options:
- Demonstrate how to build your TF model:
I download the quantize-aware training int8 model from repo goolge-research/mobilebert. The model download link is download link. - Please follow this colab page to convert the model.
-
QAT INT8 mobilebert tensorflow model: download link. Untar the file and then you can find model is in "mobilebert_squad_savedmodels/quant_saved_model".
-
Converted INT8 tflite model: download link
3. Failure after conversion
-
Model produces wrong results: FC layers zero-point != 0. These don't follow quantization spec.
-
Fail to convert the model to tflite, only tf-2.5.0 can successfully convert to INT8 tflite model. In other words, tf2.6 cannot work.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.