tensorflow / tensorflow/models
Default learning rate in TF2 SSD MobileNet V2 config file is way too high. Is it a typo?
@pkulzc is already working on this.
Since Feb 23, 2022.
- Dominant language
- Python
- Stars
- 77.7k
- Forks
- 44.8k
- PR merge metrics
- No merged PRs in 30d
Description
The learning rate set in the TF2 SSD MobileNet V2 config file is 10x higher than that of the other SSD MobileNet models. This causes loss during training to get extremely high. Is it a typo?
The default ssd_mobilenet_v2_320x320_coco17_tpu-8.config configuration has this for the learning rate:
optimizer {
momentum_optimizer: {
learning_rate: {
cosine_decay_learning_rate {
learning_rate_base: .8
total_steps: 50000
warmup_learning_rate: 0.13333
warmup_steps: 2000
}
}
momentum_optimizer_value: 0.9
Meanwhile, the FPNLite version ( ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.config ) has this:
optimizer {
momentum_optimizer: {
learning_rate: {
cosine_decay_learning_rate {
learning_rate_base: .08
total_steps: 50000
warmup_learning_rate: .026666
warmup_steps: 1000
}
}
momentum_optimizer_value: 0.9
When I train with the default values in the ssd_mobilenet_v2_320x320_coco17_tpu-8.config file, the huge learning rate throws training way off. When I change the values from .8 to .08 and .13333 to .013333, training works much better. I think whoever wrote the config file missed a decimal point.
Loss graph BEFORE changing learning rate values (the loss is way higher)

Loss graph AFTER changing learning rate values

Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.