google-deepmind / google-deepmind/optax

Add PolarExpress Variant to Muon

Open
#1,621 0 comments 0 reactions 0 assignees View on GitHub
type:feature
Dominant language
Python
Stars
2.3k
Forks
369
Avg merge
10h 15m
Merged PRs (30d)
7

Description

This has already been suggested here https://github.com/google-deepmind/optax/pull/1602.

The PolarExpress ([Amsel et al., 2025](https://arxiv.org/abs/2505.16932)) is an optimal method to compute the polar decomposition of a matrix. The authors have demonstrated that their method shows consistent improvements over other methods when used with Muon, and addressed "finite-precision issues, making it practical to use in bfloat16".

In terms of implementation, it takes a similar form to Newton-Schultz but with iteration-dependent polynomial coefficients. Hence, it is a light-weight addition to Muon and fits nicely into the existing interface.

(I promise next time I will create the issue before submitting the PR)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.