Choosing Max Learning Rate for 1cycle Scheduling
Open
- Dominant language
- Jupyter Notebook
- Stars
- 30k
- Forks
- 13.1k
- PR merge metrics
- No merged PRs in 30d
Description
In the book, the author states that we choose the max learning rate for the 1cycle scheduling the same way as we choose the optimal learning rate i.e The optimal learning rate is a bit lower than the point at which the loss starts to shoot up(about `10` times lower).

This is the curve for the CIFAR10 dataset From Exercise 8f, the max rate is chosen to be `0.05`, Shoudnt it be `0.02` or less since the loss shoots back up at `0.2 `or the like? Or is it just a matter of tuning
Thanks
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.