deepinsight / deepinsight/insightface
Train recognition: the learning rate scheduler
- Dominant language
- Python
- Stars
- 29.7k
- Forks
- 6.1k
- PR merge metrics
- No merged PRs in 30d
Description
I got confused on the learning rate scheduler in [recognition/train.py](https://github.com/deepinsight/insightface/blob/4a4b8d03fec981912fdef5b3232a37a827cbeed6/recognition/train.py#L291-L303)
Relevant code is:
```
def _batch_callback(param):
#global global_step
global_step[0]+=1
mbatch = global_step[0]
for step in lr_steps:
if mbatch==step:
opt.lr *= 0.1
print('lr change to', opt.lr)
break
_cb(param)
if mbatch%1000==0:
print('lr-batch-epoch:',opt.lr,param.nbatch,param.epoch)
```
So what's the difference between `mbatch`, `nbatch` and `global_step` here?
For me I trained with the default configurations
(where `lr_steps='100000,160000,220000`) with batch-size=110 (the maximal bs I can have, due to the GPUs Mem limitation) on a 4-GPUs machine.
From my log the learning rate changes at a very strange batch index:
```
lr change to 0.010000000000000002
lr-batch-epoch: 0.010000000000000002 7368 7
```
In my understanding the lr should decrease for the first time when `mbatch=100K`.
But in my practice it decreases at
Can someone explain the mechanism here?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.