Getting 7 as action for a game with 3 actions
- Dominant language
- Jupyter Notebook
- Stars
- 10.9k
- Forks
- 1.4k
- PR merge metrics
- No merged PRs in 30d
Description
I have been trying to train an online agent on the environment FreewayNoFrameskip-v4. Because this gym environment is not deterministic, I seeded the environment. Specifically, in [atari_lib.py](https://github.com/google/dopamine/blob/master/dopamine/discrete_domains/atari_lib.py), I added
- `env.seed(0)` after `env = gym.make(full_game_name)` in `create_atari_environment`
- `self.environment.seed(0)` at the end of the AtariPreprocessing class's `__init__` function
- `self.environment.seed(0)` at the start of the `reset` function in the AtariPreprocessing class
No other changes were made. I then used the [Batch RL codebase](https://github.com/google-research/batch_rl) to train an online agent.
In all of training, there was one instance of a 7 stored as the action (specifically the last action at the end of five iterations of training), even though Freeway only has three actions. All other stored actions were {0, 1, 2}. Any ideas what could be the cause of this? Going in and changing this one 7 to the most common action isn't a problem, but if this problem arises repeatedly, and for other games, it could be difficult to deal with.
Contributor guide
Assessment
This issue has not been assessed yet.