LR decay being flat in the end
Open
bug
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 271
Description
**Describe the bug**
LR will be flat in the end of decay, seems to be equal with the number of warming up steps.
**Steps/Code to reproduce bug**
When using, for example
TRAIN_SAMPLES=102400000
LR_WARMUP_SAMPLES=5120000
LR_DECAY_SAMPLES=$((TRAIN_SAMPLES-LR_WARMUP_SAMPLES))
--train-samples $TRAIN_SAMPLES \
--lr-decay-samples $LR_DECAY_SAMPLES \
--lr-warmup-samples $LR_WARMUP_SAMPLES \
**Additional context**
Contributor guide
Assessment
This issue has not been assessed yet.