tensorflow / tensorflow/models

freeze_variables has no effect when training with model_main_tf2.py

Open
#10,167 2 comments 0 reactions 3 assignees View on GitHub

@jch1 is already working on this.

Since Aug 3, 2021.

models:research:odapi type:bug
Dominant language
Python
Stars
77.7k
Forks
44.8k
PR merge metrics
No merged PRs in 30d

Description

I noticed the freeze_variables train config is never accounted for when training with model_main_tf2.py

Versions

  • Tensorflow 2.5
  • Commit: latest (Mon Jul 19 12:32:17 2021 -0700)
  • tf_env_collect.sh output here

1. The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/research/object_detection/model_main_tf2.py
https://github.com/tensorflow/models/blob/master/research/object_detection/model_lib_v2.py

2. Describe the bug

When I provide something in the freeze_variables config (here in train.proto), e.g. freeze_variables: ['.*conv.'], and then run training with model_main_tf2.py as in the tutorial, no variables are frozen and all layers are trained. I went through the model and training protocol configuration steps in model_lib_v2.py, in the train_loop method which is called in model_main_tf2.py, and it never seems to touch the freeze_variables option.

3. Steps to reproduce

Follow the steps of the tutorial to set up a training run for the SSD resnet50 v1 fpn model, and use the config file included with the model. Add freeze_variables: ['.*conv.'] to train_config. If you add print(detection_model.trainable_variables) to line 673 of model_lib_v2.py (right before the training begins), it outputs all variables, i.e. all variables are still trainable.

4. Expected behavior

Variables matching the provided freeze_variable regex should no longer be trainable.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.