facebookresearch / facebookresearch/coconut
Optimizer is reset every epoch rather than during training stage switch
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 189
- PR merge metrics
- No merged PRs in 30d
Description
In the paper, it is mentioned that "we also reset the optimizer state when training stages switch."
However, in
https://github.com/facebookresearch/coconut/blob/d1c10b134ad2bded92b0eca4c0a422c513a12dea/run.py#L312-L320
the optimizer is reset every epoch.
Contributor guide
Research direction
Start in run.py around lines 312-320 and compare the optimizer reset there with the paper's statement about resetting when training stages switch. Trace where the training-stage transition is handled, then verify that the optimizer is reset at stage changes rather than at every epoch; confirm the behavior with the relevant training run or tests if available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100