FEAT / Optim: Add Freeze FT and GaLore optimizer
Open
feature request
- Dominant language
- Python
- Stars
- 5.2k
- Forks
- 448
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 26
Description
As per title,
adds Freeze fintuning and
adds the GaLore optimizer from https://github.com/jiaweizzhao/GaLore
Contributor guide
Assessment
This issue has not been assessed yet.