google-deepmind / google-deepmind/mujoco_playground

Op3Joystick: trained policies barely track forward commands below 0.3 m/s (0.1 m/s → stands still)

Open
#361 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.2k
Forks
359
Avg merge
7d 3h
Merged PRs (30d)
1

Description

**What.** Policies trained on `Op3Joystick` with the default config and
`locomotion_params.brax_ppo_config("Op3Joystick")` track 0.3–0.5 m/s well but hardly move for smaller
forward commands: at 0.1 m/s they stand, at 0.2 m/s they reach about half the commanded speed in MJX, and
nothing at all when the same policy runs in C MuJoCo. "Walk slowly" is effectively unavailable to a
joystick or a planner.

**Setup.** mujoco_playground 0.2.0, mujoco/mjx 3.13, brax 0.14.2, `impl="jax"`, 100 M steps
(`num_envs=4096`), no config changes. Command held fixed (the env's command resampling disabled for the
measurement), starting from the reset pose:

| command vx (m/s) | MJX: mean forward velocity, 32 episodes × 5 s | C MuJoCo: distance / time, 5 s, same model and gains |
|---|---|---|
| 0.1 | 0.003 | 0.000 |
| 0.2 | 0.105 | 0.001 |
| 0.3 | 0.279 | 0.288 |
| 0.4 | 0.398 | 0.393 |
| 0.5 | 0.519 (128 × 10 s) | 0.513 |

At 34 M steps the C MuJoCo sweep looked the same (0.2 → 0.002 m/s), so it is not undertraining.

**Likely cause.** `tracking_lin_vel = exp(-|cmd - v|² / tracking_sigma)` with `tracking_sigma=0.25`:
standing still under a 0.2 m/s command still earns `exp(-0.04/0.25) ≈ 0.85` of the tracking reward
(0.96 at 0.1 m/s), while stepping costs `action_rate`, `energy`, `torques` and risks termination. For
small commands, standing (or shuffling) is near-optimal, and the half-speed gait at 0.2 m/s is fragile
enough to disappear under a small sim-to-sim change.

**Possible fixes** (happy to test and PR one): a smaller `tracking_sigma` or one scaled by the command
magnitude, a minimum non-zero speed in `sample_command` alongside the explicit zero-command share, or a
command-magnitude curriculum. The table above is a cheap acceptance test for any of them.

Contributor guide

Open the contributing guide

Research direction

Start with the Op3Joystick environment, locomotion_params.brax_ppo_config("Op3Joystick"), and the sample_command path. Reproduce the reported MJX and C MuJoCo sweeps with command resampling disabled, then compare candidate reward or command-sampling changes. Done means small commands produce meaningful forward motion without regressing the 0.3–0.5 m/s results, using the table as the acceptance test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, robotics
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.