tensorflow / tensorflow/model-optimization

Pruning Transfer Learning Models

Open
#749 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.6k
Forks
349
Avg merge
3d 2h
Merged PRs (30d)
1

Description

Essentially, I want to perform pruning to my transfer learning model.

I used efficientnetb0 as the base model for transfer learning method.
I am now following the guide to pruning

import tensorflow_model_optimization as tfmot

prune_low_magnitude = tfmot.sparsity.keras.prune_low_magnitude

# Compute end step to finish pruning after 2 epochs.
batch_size = 32
epochs = 25

end_step = np.ceil(len(training_set) / batch_size).astype(np.int32) * epochs

# Define model for pruning.
pruning_params = {
      'pruning_schedule': tfmot.sparsity.keras.PolynomialDecay(
                                        initial_sparsity = 0.40,                                                                 
                                        final_sparsity = 0.90,                                                                   
                                        begin_step = 0,                                                                
                                        end_step = end_step
                                        )
                  }

model_for_pruning = prune_low_magnitude(
                         efficientnetb0_transfer_model, **pruning_params)

# `prune_low_magnitude` requires a recompile.
efficientnetb0_transfer_model_for_pruning.compile(optimizer=optim,
              loss='categorical_crossentropy',
              metrics=['accuracy'])

efficientnetb0_transfer_model_for_pruning.summary()

But I'm getting the following error:

ValueError: Please initialize Prunewith a supported layer. Layers should either be supported by the PruneRegistry (built-in keras layers) or should be aPrunableLayerinstance, or should has a customer definedget_prunable_weights method. You passed: <class 'tensorflow.python.keras.layers.preprocessing.image_preprocessing.Rescaling'>

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked pruning-with-Keras guide and the supplied EfficientNetB0 transfer-learning code. Investigate how prune_low_magnitude handles the Keras Rescaling layer and whether that layer is supported; done means establishing a documented, working path for pruning this model or clearly identifying the unsupported configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.