tensorflow / tensorflow/model-optimization
TFLite quantization, the int tflite model is much slower than the float tflite model
Open
@daverim is already working on this.
Since Aug 27, 2021.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 349
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 1
Description
I converted Pytorch model to onnx model, and then to tensorflow model, I convert tensorflow model to float tflite model and int tflite model(with post training quantization). Currently, my int tflite model is much slower than my float tflite model.
BTW, my model is based on transformer architecture.
Here are the comparisons:
Pytorch: 0.04s, 0 mse, 439 MBs
ONNX-Ori: 0.02s, 2.062e-12 mse, 437 MBs
ONNX-Opt: 0.02s, 2.062e-12 mse, 437 MBs
TFLite: 0.21s, 5.496e-12 mse, 551 MBs
Quantized TFLite: 1.42s, 223.7 mse, 138 MBs
Could you please tell me what the reason is?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.