google-research / google-research/big_vision

Add other custom optimizers

Open
#32 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
3.5k
Forks
227
PR merge metrics
No merged PRs in 30d

Description

I noticed that there are [no other choices of optimizers other than scale_by_adafactor()](https://github.com/google-research/big_vision/blob/c62890a3e4487b1d6751794b090138b9da5d18e1/big_vision/optax.py#L157)

This github issue would serve as a placeholder for other optimizers such as`Adam#Lion` or others in the future.

TODO:
- [ ] Add `Adam#Lion` jax code

Contributor guide

Open the contributing guide

Research direction

Start in big_vision/optax.py at the existing scale_by_adafactor() implementation linked in the issue, then review how optimizer choices are exposed. Add the requested Adam#Lion JAX code and verify that Lion is available as another optimizer choice; the issue does not name a test file or test command.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.