tensorflow / tensorflow/model-optimization
Tflite full Integer quantization bug.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 349
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 1
Description
Hi, I am trying to quantize my own tf-keras model for NNAPI delegate. First the origin tf-keras model size is 5.6MB,and quantized int8-tflite model size is about 1.8M, that is not 4X times. And more important is : i am trying to run the quantized int8-tflite model on this test benchmark (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark), the result is : use_nnapi=true inference time is longer than use_nnapi=false(202883us / 45882.2us). And i test the quantized model in tensorflow hub, which inference time are fine on my mobile device,(3x about time reduce.), but I quantize the original SaveModel format model in tensorflow hub according to Tutorials (https://www.tensorflow.org/lite/performance/post_training_quantization). the result is same as my own quantized int8-tflite model. such as moblenet_v2_130_224: (size:21.6/6.3)(time(nnapi true/false):104323 / 42137.5 ). The other model is the same as mine. Meanwhile , I quantized mnist example, finally get the same result(nnapi=true , inference time is longer than nnapi=false.). and I want to quantize the given example of quantized model and original model, but which are not SaveModel format , so I can not load the model to process. So , are there problems in my quantization process?
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
- OS Platform and Distribution :macos BigSur 11.5.1
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: DM-AI-V1.1(haveing one NPU)
- TensorFlow installed from (source or binary):
- TensorFlow version (use command below): 2.5
- Python version: 3.7
- Bazel version (if compiling from source):
- GCC/Compiler version (if compiling from source):
- CUDA/cuDNN version:
- GPU model and memory:
Thx.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.