tensorflow / tensorflow/model-optimization

How to get quantized weights from QAT model?

Open
#626 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.6k
Forks
349
Avg merge
3d 2h
Merged PRs (30d)
1

Description

Hi all. I've recently trained a keras implementation of ssd-keras. I've managed to run QAT training on the model and got desired the accuracy. I wanted to get the quantised weights from the QAT model saved as a H5 model. There's no support or documentation regarding this in the tensorflow website. How can I get the quantised weights from the saved model after QAT?
I tried converting it to TFLite but it is not converting due to a custom layer in model definition. So it would be helpful if I can get the quantised weights alone from the saved model. Here's the code snippet for my QAT training. I am using TF 2.3.

`quantize_scope = tfmot.quantization.keras.quantize_scope

def apply_quantization_to_dense(layer):
if 'priorbox' in layer.name:
return layer

if isinstance(layer,tf.keras.layers.Concatenate) or isinstance(layer, tf.keras.layers.Reshape) or isinstance(layer,tf.keras.layers.Lambda):
return layer

return tfmot.quantization.keras.quantize_annotate_layer(layer)

annotated_model = tf.keras.models.clone_model(
model,
clone_function=apply_quantization_to_dense,
)

with quantize_scope({'AnchorBoxes': AnchorBoxes}):
quant_aware_model = tfmot.quantization.keras.quantize_apply(annotated_model)

quant_aware_model.summary()
quant_aware_model.compile(optimizer=adam, loss=ssd_loss.compute_loss)
quant_aware_model.fit_generator(train_generator, epochs=424, steps_per_epoch=1000,
callbacks=callbacks, validation_data=val_generator,
validation_steps=100, initial_epoch=414)`

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the QAT setup in the issue, especially quantize_scope, quantize_apply, the custom AnchorBoxes layer, and the saved H5 model path. Determine whether quantized weights can be retrieved directly or whether conversion through TFLite is required, then document the supported workflow and how the custom layer affects conversion. Done means a reproducible answer for the TensorFlow 2.3 setup described.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.