google-deepmind / google-deepmind/mujoco

[MJX] jax.lax.while_loop in solver.py prevents computation of backward gradients

Open
#2,259 15 comments 5 reactions 2 assignees Claimed by @btaba View on GitHub
enhancement good first issue MJX
Dominant language
C++
Stars
15.2k
Forks
1.8k
Avg merge
10d 16h
Merged PRs (30d)
25

Description

### The feature, motivation and pitch

## Problem
The solver's `jax.lax.while_loop` implementation prevents gradient computation through the environment step during gradient based trajectory optimization. This occurs in the [solver implementation](https://github.com/google-deepmind/mujoco/blob/300450f8b8441e5f00abc1105089a23f50d7147c/mjx/mujoco/mjx/_src/solver.py#L570) when iterations > 1.

Error encountered with `jax.jit` compiled grad function:
```
ValueError: Reverse-mode differentiation does not work for lax.while_loop or lax.fori_loop with dynamic start/stop values.
```

Current workaround of using `opt.iteration=1` leads to potentially inaccurate simulation and gradients.

## Proposed Solution
Add an option to set a fixed iteration count (e.g., 4) that would be compatible with reverse-mode differentiation using either `lax.scan` or `lax.fori_loop` with static bounds.

### Alternatives

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.