PolyakSGD should take an optional parameter f_star
Open
enhancement
good first issue
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 76
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 1
Description
the SPS-SGD step-size (Loizou version) is
$$
\frac{f_i(x) - \inf_z f_i(z)}{\|\nabla f(x)\|^2}
$$
The current code assumes that $\inf_z f_i(z)=0$ (without saying anything in the documentation). Instead, I believe it should take an optional argument called (say) `f_star`.
Contributor guide
Assessment
This issue has not been assessed yet.