tensorflow / tensorflow/model-optimization
CONV2D, DENSE layer with default QAT can't generate correct int8 quantized nodes in TFLite model
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 349
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 1
Description
Describe the bug
when converting a MNIST model using QAT, quantize selective CON2D or DENSE layer with tfmot.quantization.keras.quantize_annotate_layer can't generate correct int8 quantized TFLite model.
check colab for bug reproduce
QAT only CON2D layer:
in generated tflite model (quantized_conv2d_mnist.tflite), nodes following CON2D layer been quantized, but CONV2D layer is not quantized.
QAT only Dense layer:
in generated tflite model (quantized_dense_mnist.tflite), nodes following DENSE layer been quantized, but DENSE layer is not quantized.
System information
TensorFlow version (installed from source or binary): 2.4.0-dev20200819
TensorFlow Model Optimization version (installed from source or binary): 0.4.1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked Colab reproduction and inspect the generated quantized_conv2d_mnist.tflite and quantized_dense_mnist.tflite models. Compare the CONV2D and DENSE nodes with the following quantized nodes using the TensorFlow Model Optimization 0.4.1 and TensorFlow 2.4.0-dev20200819 setup. Done means selectively annotated layers produce the expected int8 TFLite nodes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100