tensorflow / tensorflow/model-optimization
Pruning Transfer Learning Models
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 349
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 1
Description
Essentially, I want to perform pruning to my transfer learning model.
I used efficientnetb0 as the base model for transfer learning method.
I am now following the guide to pruning
import tensorflow_model_optimization as tfmot
prune_low_magnitude = tfmot.sparsity.keras.prune_low_magnitude
# Compute end step to finish pruning after 2 epochs.
batch_size = 32
epochs = 25
end_step = np.ceil(len(training_set) / batch_size).astype(np.int32) * epochs
# Define model for pruning.
pruning_params = {
'pruning_schedule': tfmot.sparsity.keras.PolynomialDecay(
initial_sparsity = 0.40,
final_sparsity = 0.90,
begin_step = 0,
end_step = end_step
)
}
model_for_pruning = prune_low_magnitude(
efficientnetb0_transfer_model, **pruning_params)
# `prune_low_magnitude` requires a recompile.
efficientnetb0_transfer_model_for_pruning.compile(optimizer=optim,
loss='categorical_crossentropy',
metrics=['accuracy'])
efficientnetb0_transfer_model_for_pruning.summary()
But I'm getting the following error:
ValueError: Please initialize Prunewith a supported layer. Layers should either be supported by the PruneRegistry (built-in keras layers) or should be aPrunableLayerinstance, or should has a customer definedget_prunable_weights method. You passed: <class 'tensorflow.python.keras.layers.preprocessing.image_preprocessing.Rescaling'>
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked pruning-with-Keras guide and the supplied EfficientNetB0 transfer-learning code. Investigate how prune_low_magnitude handles the Keras Rescaling layer and whether that layer is supported; done means establishing a documented, working path for pruning this model or clearly identifying the unsupported configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100