google / google/dopamine

Why not separate agent and environment?

Open
#110 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
10.9k
Forks
1.4k
PR merge metrics
No merged PRs in 30d

Description

The different parts in dopamine are connected to each other. It really makes the whole project hard to modify or extend.
For example, the Agent part needs to store transitions in Agent itself. If I want to accelerate the training process by implement multi-env, I have to change the Agent part in order to make this happen, such as stack observations in each environment and remove this part from Agent.
Other things are also connected to each other. Replay buffer can be built in the Runner and only send tensor to Agent. Network structures are declared in `dqn_agent.py` and `atari_lib.py`.
It's easy to run dopamine tested experiments. But this project is hard to be modified in order to use in other games. And only using one environment to collect data makes the progress slow.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.