google / google/brax

Reconstructing scene from qpos and qvel

Open
#345 14 comments 2 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
3.2k
Forks
349
PR merge metrics
No merged PRs in 30d

Description

I've mentioned this briefly in #157, but I believe this deserves a separate thread.

In certain applications like Go-Explore (https://www.nature.com/articles/s41586-020-03157-9), it's critical for the user to be able to reset the environment to the state corresponding to the gathered observation, i.e., qpos and qvel.

In my experiments with Brax v2, I haven't been able to successfully reconstruct the simulator state from qpos and qvel alone--I suspect the 2nd order information is also necessary. According to MuJoCo docs:

> class mujoco_py.MjSimState
>
> Represents a snapshot of the simulator’s state.
>
> This includes time, qpos, qvel, act, and udd_state.
>
> Attributes
>
> Methods

Ignoring `time` and `udd_state`, `act` needs to be taken into account. Not sure by what mechanism MuJoCo does this, but we can gather that `qpos`, `qvel`, and `act` should be necessary and sufficient for reconstructing the state.

Current workaround for this is to actually store the states (i.e. in a batched instance of `State` class) gathered during exploration and to use that as the argument for `step()`. But this comes at a huge memory overhead.

Any idea whether this will be made possible?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.