google / google/flax

`nnx.vmap` use the same random key `rngs` inside nnx.Module across vectorization.

Open
#4,195 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
7.3k
Forks
833
Avg merge
5h 11m
Merged PRs (30d)
5

Description

Hi,

If I run something like:
```
def inference_fn(model, x):
y = model(input)
return y

ys = nnx.vmap(inference_fn, in_axes=(None, 0))(model, xs)
```
the random key that is used across the vectorization is unique. It means that stochastic functions will have the same behavior across the batch.

- Is this the intended behavior for nnx.vmap when dealing with stochastic models?
- If yes, is there a recommended way to ensure independent random keys are used for each batch element when using nnx.vmap?

Thanks!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.