Why not separate agent and environment?
- Dominant language
- Jupyter Notebook
- Stars
- 10.9k
- Forks
- 1.4k
- PR merge metrics
- No merged PRs in 30d
Description
The different parts in dopamine are connected to each other. It really makes the whole project hard to modify or extend.
For example, the Agent part needs to store transitions in Agent itself. If I want to accelerate the training process by implement multi-env, I have to change the Agent part in order to make this happen, such as stack observations in each environment and remove this part from Agent.
Other things are also connected to each other. Replay buffer can be built in the Runner and only send tensor to Agent. Network structures are declared in `dqn_agent.py` and `atari_lib.py`.
It's easy to run dopamine tested experiments. But this project is hard to be modified in order to use in other games. And only using one environment to collect data makes the progress slow.
Contributor guide
Assessment
This issue has not been assessed yet.