google-deepmind / google-deepmind/optax
Updates dtype do not need to match params dtype but only grads dtype a priori
Open
type:feature
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 369
- Avg merge
- 10h 15m
- Merged PRs (30d)
- 7
Description
Revise #1060 in light of the discussion of #1039.
Namely, make tests ensure that dtype of grads is preserved (not necessarily the same as the dtype of params for e.g. mixed precision training).
Revise the patch #1060 to see if the initialization of the dtypes in the states are too stringent or not.
Contributor guide
Assessment
This issue has not been assessed yet.