google / google/dopamine

Striping out the TimeLimit wrapper

Open
#118 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
10.9k
Forks
1.4k
PR merge metrics
No merged PRs in 30d

Description

The comment said:
> Strip out the TimeLimit wrapper from Gym, which caps us at 100k frames. We handle this time limit internally instead, which lets us cap at 108k frames (30 minutes).

https://github.com/google/dopamine/blob/85fa5c27de4f871c1cc5131ec3cfb4c0a1f87a61/dopamine/discrete_domains/atari_lib.py#L74-L80

In the general setting, The `max_steps_per_episode` is 27000 and the number of frames is **108k** = 27000*(frameskip=4).
I think that the default TimeLimit caps at **400k** = (frameskip=4) * 100000 instead of **100k** in the Gym. I wondered if we didn't use other wrappers over the environment, does it need to strip out the TimeLimit (**400k>108k**)? Besides, I am not sure about the saving and restoring part.
Please let me know if I am wrong.

The register part in gym:
https://github.com/openai/gym/blob/c33cfd8b2cc8cac6c346bc2182cd568ef33b8821/gym/envs/__init__.py#L653-L659

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.