tensorflow / tensorflow/model-optimization
How to user quantize to imporve inference performance on tensorflow-serving?
@nutsiepully is already working on this.
Since Aug 7, 2020.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 349
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 1
Description
Prior to filing: check that this should be a bug instead of a feature request. Everything supported, including the compatible versions of TensorFlow, is listed in the overview page of each technique. For example, the overview page of quantization-aware training is here. An issue for anything not supported should be a feature request.
Describe the bug
I use the "Quantization aware training in Keras example"
and save 4 models: model(model without quantization aware training), q_aware_model(model with quantization aware training), quantized_tflite_model(TFLite model with quantization aware training ), float_tflite_model(TFLite model without quantization), and depoly the models with tensorflow-serving,but I got the same performance. I dont konw the reason. How can I use the quantized model to imporve inference performance on tensorflow-serving?
System information
TensorFlow version (installed from source or binary): nightly
TensorFlow Model Optimization version (installed from source or binary): binary
Python version: python3.8
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.