Farama-Foundation / Farama-Foundation/Arcade-Learning-Environment
ALE v0.6 differences in start state
- Dominant language
- C++
- Stars
- 2.5k
- Forks
- 477
- Avg merge
- 11h 11m
- Merged PRs (30d)
- 3
Description
There was an issue raised (https://github.com/openai/gym/issues/1777) which describes differences between `v0.5.2` and `v0.6.0` of the ALE. I traced some of the issues to this commit https://github.com/mgbellemare/Arcade-Learning-Environment/commit/7bff96b4b64edcffbeb2d9bb83b1685ab506ea2b#diff-d9d868097a7403416e6ef352d95dc4feR178 which changes how `StellaEnvironment::softReset` works.
The `RESET` action is called `m_num_reset` times which leads to a different starting state for the agent. Perhaps this was intended behaviour in `StellaEnvironment::reset` but has ill-intended consequences in `StellaEnvironment::softReset`.
For example, here are the starting states for Ms. Pacman in ALE `v0.5.2` and `v0.6.0`. Note if you emulate **one** `RESET` action then we get the `v0.5.2` starting state.
### Ms. Pacman, ALE v0.5.2

### Ms. Pacman, ALE v0.6.0

You can see the subtle changes between these two frames (e.g., the colour of ghosts in jail).
I haven't looked into why we repetitively call `RESET`. Should this be something that is investigated further? It wouldn't seem that this should affect asymptotic performance.
Contributor guide
Research direction
Start with StellaEnvironment::reset and StellaEnvironment::softReset, then inspect the linked ALE commit and the RESET action handling. Compare the reset counts and Ms. Pacman starting states between ALE v0.5.2 and v0.6.0; done means the intended soft-reset behavior is established and the version difference is resolved or clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- game-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100