tensorflow / tensorflow/model-optimization

quantization not happening?

Open
#829 4 comments 0 reactions 1 assignee View on GitHub

@Xhark is already working on this.

Since Sep 27, 2021.

Dominant language
Python
Stars
1.6k
Forks
349
Avg merge
3d 2h
Merged PRs (30d)
1

Description

Hello,
I have tried to make my model comaptible with QAT, according to your guideline.
I started with defining a QuantizeConfig class:

LastValueQuantizer = tfmot.quantization.keras.quantizers.LastValueQuantizer
MovingAverageQuantizer = tfmot.quantization.keras.quantizers.MovingAverageQuantizer

class DefaultConv2DQuantizeConfig(tfmot.quantization.keras.QuantizeConfig):
    # Configure how to quantize weights.
    def get_weights_and_quantizers(self, layer):
      return [(layer.kernel, LastValueQuantizer(num_bits=4, symmetric=True, narrow_range=False, per_axis=False))]

    # Skip quantizing activations.
    def get_activations_and_quantizers(self, layer):
      return []

    def set_quantize_weights(self, layer, quantize_weights):
      # Add this line for each item returned in `get_weights_and_quantizers`
      # , in the same order
      layer.kernel = quantize_weights[0]

    def set_quantize_activations(self, layer, quantize_activations):
      # Empty since `get_activaations_and_quantizers` returns
      # an empty list.
      return

    # Configure how to quantize outputs (may be equivalent to activations).
    def get_output_quantizers(self, layer):
      return [MovingAverageQuantizer(num_bits=4, symmetric=False, narrow_range=False, per_axis=False)]

    def get_config(self):
      return {}

Then I applied the quantization:

def apply_mix_precision_QAT2(layer):
  # if isinstance(layer, tf.keras.layers.Dense):
  if isinstance(layer, tf.keras.layers.Conv2D):
    return tfmot.quantization.keras.quantize_annotate_layer(layer, quantize_config=DefaultConv2DQuantizeConfig())
  return layer

annotated_model = tf.keras.models.clone_model(model,clone_function=apply_mix_precision_QAT2)
with tfmot.quantization.keras.quantize_scope({'DefaultConv2DQuantizeConfig': DefaultConv2DQuantizeConfig}):
  model = tfmot.quantization.keras.quantize_apply(annotated_model)

Finally, I chose one of the Conv2D layers, which evidently was quantized:
image

and looked at its weights, and it appears they have not been quantized to a 4bit encoding, like I specified in the QuantizeConfig:
image

Do I have an error? Or is there another way to check whether the layer was quantized?
Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.