google / google/brax

Significant changes in scores with the v0.0.11 physics

Open
#184 4 comments 2 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
3.2k
Forks
349
PR merge metrics
No merged PRs in 30d

Description

### Context:
I am running my implemention of the PGA-MAP-Elites ([see paper](https://hal.archives-ouvertes.fr/hal-03135723v2/file/PGA_MAP_Elites_GECCO.pdf)) in brax halfcheetah.

I obtained good and consistent results with the old `v0.0.10` physics in `halfcheetah` (and in all other environments too), ranging from 6000 to 7000 at the very end of the run.

### Problem description:
Since the `v0.0.11` version, these results have changed a lot, here are some points that I want to mention:

- 1) At the beginning of the runs, scores are significantly higher and have more variance. With the old physics, the first scores that I plotted ranged from 2000 to 3000 maximum and never went out of these bounds. With the new physics, starting scores range from 4000 to 7000 (which is more than what I obtained for a full run using old physics). Of course these absolute values are not relevant but the sudden change is.

- 2) For a full run, there is way less amplitude in the score evolution. Final scores with new physics are still ranging from 6000 to 7000 but now they start way higher.

- 3) I want to emphasize that obtaining 7000 at the very beginning of a run seems problematic since for another seed with the same parameters I could not be reaching 7000 even at the end of the run.

- 4) It makes comparisons with old results and brax oriented papers irrelevant, at least in halfcheetah.

### Additional details:
All parameters are kept equal for these comparisons, only the physics changes (I made use of `legacy_spring=True` to switch between versions).
The results reported here take into account multiple seeds for each version, by that I mean that I see a very significant change that cannot be due to statistical noise.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.