deepdrive / deepdrive/deepdrive-zero
Use NN with better sequential modeling ability
- Dominant language
- Python
- Stars
- 40
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
We currently use an MLP for the actor and critic networks in PPO. This _should_ be fine so long as the environment is fully observable, but presents scaling issues when dealing with variable numbers of agents in the scene.[ See this TODO for context.](https://github.com/deepdrive/deepdrive-2d/blob/d4d8fcc40842cffb7473663e1181394ac947061d/deepdrive_2d/envs/agent.py#L453-L455)
Also, we eventually will need to introduce partial observability of other agents to simulate the occlusion that occurs in real-world vehicles - so having a NN capable of some type of memory / sequential modeling will be necessary eventually. OpenAI used LSTMs with PPO for Dota2, so there may be some info on doing that. Also transformers seemingly handle long term dependencies and sequential modeling more efficiently, and have been successfully applied to RL with Deepmind's GTrXL work.
[Note: We should move to Pytorch first](https://github.com/crizCraig/spinningup/issues/1) to make NN modifications much more tractable, fun, sane, pleasurable, easy, fast etc...
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.