google-deepmind / google-deepmind/optax

RMS transforms produce NaN on float16 zero-gradient steps

Open
#1,784 0 comments 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Python
Stars
2.3k
Forks
369
Avg merge
10h 15m
Merged PRs (30d)
7

Description

### Description

`scale_by_rms` and `scale_by_stddev` produce `NaN` updates on the first zero-gradient step when the updates and state use `float16`. This also affects `optax.rmsprop`.

The default `eps=1e-8` is evaluated in float16 during the denominator calculation and underflows to zero. The result is an infinite scale multiplied by a zero gradient.

### Reproduction

```python
import jax.numpy as jnp
import optax

for factory in (optax.scale_by_rms, optax.scale_by_stddev):
tx = factory()
grads = jnp.zeros((4,), dtype=jnp.float16)
updates, _ = tx.update(grads, tx.init(grads))
print(factory.__name__, updates)
```

Current output:

```
scale_by_rms [nan nan nan nan]
scale_by_stddev [nan nan nan nan]
```

The issue occurs with both values of `eps_in_sqrt` and `bias_correction`. It also occurs when an ArrayLike epsilon has already underflowed, for example `jnp.asarray(1e-8, dtype=jnp.float16)`.

### Expected behavior

A zero gradient should produce a finite, zero update while preserving the float16 update dtype.

### Proposed fix

Compute the variance and denominator in at least float32, floor a zero denominator to the smallest positive normal value of that computation dtype, and cast the final update back to the gradient dtype. Add JIT-compiled regression coverage for both RMS transforms, both epsilon placements, bias correction on/off, and scalar/ArrayLike epsilon values.

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied reproduction for optax.scale_by_rms and optax.scale_by_stddev, then inspect those transforms and the optax.rmsprop path. Add JIT-compiled regression coverage for both epsilon placements, bias correction on and off, and scalar or ArrayLike epsilon values; done means zero float16 gradients yield finite zero updates with the float16 dtype preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.