google / google/dopamine

Dopamine duplicate Gym functionality

Open
#82 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
10.9k
Forks
1.4k
PR merge metrics
No merged PRs in 30d

Description

I spent a lot of time trying to understand the colab cartpole Gym example so to apply to a custom discrete Gym environment, which is similar to the cartpole gym environment and works fine with a Keras RL agent. I noticed that dopamine is using gym_lib.py in addition to the actual gym environment. For example, gym_lib.py contains variables already defined in the gym cartpole environment file. such as:

dopamine/discrete_domains/gym_lib.py:
```
CARTPOLE_MIN_VALS = np.array([-2.4, -5., -math.pi/12., -math.pi*2.])
CARTPOLE_MAX_VALS = np.array([2.4, 5., math.pi/12., math.pi*2.])
gin.constant('gym_lib.CARTPOLE_OBSERVATION_SHAPE', (4, 1))
gin.constant('gym_lib.CARTPOLE_OBSERVATION_DTYPE', tf.float32)
```

gym/envs/classic_control/cartpole.py:
```
self.x_threshold = 2.4
self.observation_space = spaces.Box(-high, high, dtype=np.float32)

```

This is confusing.
OpenaAI gym encompass all the code necessary to create a complete environment object. With all the necessary plumbing//functions/variables, etc.. This approach is difficult to understand and build on.
Would it be possible to make gym a drop-in for dopamine? This would greatly simplify and speed up dopamine adoption.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.