ageron / ageron/handson-ml2

Choosing Max Learning Rate for 1cycle Scheduling

Open
#289 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
30k
Forks
13.1k
PR merge metrics
No merged PRs in 30d

Description

In the book, the author states that we choose the max learning rate for the 1cycle scheduling the same way as we choose the optimal learning rate i.e The optimal learning rate is a bit lower than the point at which the loss starts to shoot up(about `10` times lower).

![image](https://user-images.githubusercontent.com/45713796/95977956-257ef700-0e37-11eb-998a-524fe5b697f0.png)
This is the curve for the CIFAR10 dataset From Exercise 8f, the max rate is chosen to be `0.05`, Shoudnt it be `0.02` or less since the loss shoots back up at `0.2 `or the like? Or is it just a matter of tuning

Thanks

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.