tensorflow / tensorflow/model-optimization

Missing custom input quantization

Open
#589 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request
Dominant language
Python
Stars
1.6k
Forks
349
Avg merge
3d 2h
Merged PRs (30d)
1

Description

Currently implemented quantization aware training algorithm gives the chance to test different types of quantization on a model by implementing a custom Quantizer and annotating layers with custom QuantizeConfig classes, but this capability is highly limited by the fact that a default 8bit quantization layer is added after the input layer of the model.
This extra layer will always quantize inputs to 8bits integer, thus it is not possible to explore other types of quantization which could require a different input format.

The quantize_model and quantize_apply methods should provide a more customizable interface that exposes the possibility to implement a user-defined input quantization which will replace the standard 8bit quantization layer. With such an interface it would be possible to enhance the design space exploration capabilities of the QAT algorithm.

System information

  • TensorFlow version: 2.5.0-dev20201104
  • TensorFlow Model Optimization version: 0.5.0
  • Are you willing to contribute it: Yes

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names quantize_model and quantize_apply as entry points; read their implementations and trace where the default 8-bit input quantization layer is inserted. Done means these methods expose a user-defined input quantization path that can replace that layer while preserving existing behavior for standard models.

Written by the indexing model from the issue text.

Assessment

Tech stack
keras, python, tensorflow
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.