google-deepmind / google-deepmind/optax
Add PolarExpress Variant to Muon
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 369
- Avg merge
- 10h 15m
- Merged PRs (30d)
- 7
Description
This has already been suggested here https://github.com/google-deepmind/optax/pull/1602.
The PolarExpress ([Amsel et al., 2025](https://arxiv.org/abs/2505.16932)) is an optimal method to compute the polar decomposition of a matrix. The authors have demonstrated that their method shows consistent improvements over other methods when used with Muon, and addressed "finite-precision issues, making it practical to use in bfloat16".
In terms of implementation, it takes a similar form to Newton-Schultz but with iteration-dependent polynomial coefficients. Hence, it is a light-weight addition to Muon and fits nicely into the existing interface.
(I promise next time I will create the issue before submitting the PR)
Contributor guide
Assessment
This issue has not been assessed yet.