google / google/brax

Autoreset behavior

Open
#174 14 comments 4 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
3.2k
Forks
349
PR merge metrics
No merged PRs in 30d

Description

I've been digging into Brax as a potential alternative to some modified dm_control enviornments I've been using and am really loving the speedup! That said, I feel like I've run into a major issue using the environments in RL and was looking for some guidance.

Basically, my environments are all partially-observable domains built off of "ant". A lot of the conditions are randomized per-episode (e.g., ant/target starting positions). I've been using the "create_gym_env" feature to work with my PyTorch agents, but I noticed a big potential issue.

At first glance, the AutoResetWrapper seemed to do what standard gym VectorEnvs do, but in reality, it's not really "resetting" the environments (with a new seed) but instead just setting them back to a cached first state. So the randomization of start conditions I do only applies across the whole batch of environments, and then for the entire training process, each individual environment is the same as it was before.

Is there a way to _actually_ reset individual environments within a batch?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.