google / google/dopamine

Getting 7 as action for a game with 3 actions

Open
#170 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
10.9k
Forks
1.4k
PR merge metrics
No merged PRs in 30d

Description

I have been trying to train an online agent on the environment FreewayNoFrameskip-v4. Because this gym environment is not deterministic, I seeded the environment. Specifically, in [atari_lib.py](https://github.com/google/dopamine/blob/master/dopamine/discrete_domains/atari_lib.py), I added

- `env.seed(0)` after `env = gym.make(full_game_name)` in `create_atari_environment`
- `self.environment.seed(0)` at the end of the AtariPreprocessing class's `__init__` function
- `self.environment.seed(0)` at the start of the `reset` function in the AtariPreprocessing class

No other changes were made. I then used the [Batch RL codebase](https://github.com/google-research/batch_rl) to train an online agent.

In all of training, there was one instance of a 7 stored as the action (specifically the last action at the end of five iterations of training), even though Freeway only has three actions. All other stored actions were {0, 1, 2}. Any ideas what could be the cause of this? Going in and changing this one 7 to the most common action isn't a problem, but if this problem arises repeatedly, and for other games, it could be difficult to deal with.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.