facebookresearch / facebookresearch/coconut

Optimizer is reset every epoch rather than during training stage switch

Open
#39 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
189
PR merge metrics
No merged PRs in 30d

Description

In the paper, it is mentioned that "we also reset the optimizer state when training stages switch."

However, in
https://github.com/facebookresearch/coconut/blob/d1c10b134ad2bded92b0eca4c0a422c513a12dea/run.py#L312-L320
the optimizer is reset every epoch.

Contributor guide

Open the contributing guide

Research direction

Start in run.py around lines 312-320 and compare the optimizer reset there with the paper's statement about resetting when training stages switch. Trace where the training-stage transition is handled, then verify that the optimizer is reset at stage changes rather than at every epoch; confirm the behavior with the relevant training run or tests if available.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.