google-deepmind / google-deepmind/mujoco
[MJX] jax.lax.while_loop in solver.py prevents computation of backward gradients
- Dominant language
- C++
- Stars
- 15.2k
- Forks
- 1.8k
- Avg merge
- 10d 16h
- Merged PRs (30d)
- 25
Description
### The feature, motivation and pitch
## Problem
The solver's `jax.lax.while_loop` implementation prevents gradient computation through the environment step during gradient based trajectory optimization. This occurs in the [solver implementation](https://github.com/google-deepmind/mujoco/blob/300450f8b8441e5f00abc1105089a23f50d7147c/mjx/mujoco/mjx/_src/solver.py#L570) when iterations > 1.
Error encountered with `jax.jit` compiled grad function:
```
ValueError: Reverse-mode differentiation does not work for lax.while_loop or lax.fori_loop with dynamic start/stop values.
```
Current workaround of using `opt.iteration=1` leads to potentially inaccurate simulation and gradients.
## Proposed Solution
Add an option to set a fixed iteration count (e.g., 4) that would be compatible with reverse-mode differentiation using either `lax.scan` or `lax.fori_loop` with static bounds.
### Alternatives
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.