tensorflow / tensorflow/model-optimization

Tflite full Integer quantization bug.

Open
#775 8 comments 0 reactions 1 assignee View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.6k
Forks
349
Avg merge
3d 2h
Merged PRs (30d)
1

Description

Hi, I am trying to quantize my own tf-keras model for NNAPI delegate. First the origin tf-keras model size is 5.6MB,and quantized int8-tflite model size is about 1.8M, that is not 4X times. And more important is : i am trying to run the quantized int8-tflite model on this test benchmark (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark), the result is : use_nnapi=true inference time is longer than use_nnapi=false(202883us / 45882.2us). And i test the quantized model in tensorflow hub, which inference time are fine on my mobile device,(3x about time reduce.), but I quantize the original SaveModel format model in tensorflow hub according to Tutorials (https://www.tensorflow.org/lite/performance/post_training_quantization). the result is same as my own quantized int8-tflite model. such as moblenet_v2_130_224: (size:21.6/6.3)(time(nnapi true/false):104323 / 42137.5 ). The other model is the same as mine. Meanwhile , I quantized mnist example, finally get the same result(nnapi=true , inference time is longer than nnapi=false.). and I want to quantize the given example of quantized model and original model, but which are not SaveModel format , so I can not load the model to process. So , are there problems in my quantization process?

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
  • OS Platform and Distribution :macos BigSur 11.5.1
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: DM-AI-V1.1(haveing one NPU)
  • TensorFlow installed from (source or binary):
  • TensorFlow version (use command below): 2.5
  • Python version: 3.7
  • Bazel version (if compiling from source):
  • GCC/Compiler version (if compiling from source):
  • CUDA/cuDNN version:
  • GPU model and memory:

Thx.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.