pytorch / pytorch/rl

[Feature Request] Prebuilt models

Open
#1,425 5 comments 0 reactions 1 assignee View on GitHub

@vmoens is already working on this.

Since Jul 28, 2023.

enhancement
Dominant language
Python
Stars
3.6k
Forks
487
Avg merge
23h 50m
Merged PRs (30d)
209

Description

Motivation

It can be unnecessary tedious to write actors (and value) models.
A possible solution would be to have pre-built TensorDictSequence subclasses.
Here are a few examples:

from torchrl.modules import Architectures

# builds an actor with a TanhNormal distribution with [-1, 1] bound
actor = TanhActor(
   out_features=4,
   backbone=Architectures.MLP
)
# builds an actor with a Categorical distribution with 8 degrees of freedom bound
actor = CategoricalActor(
   out_features=4,
   backbone=Architectures.MLP
)
# builds an actor with a Categorical distribution for pixels with 8 degrees of freedom bound
actor = CategoricalActor(
   out_features=4,
   backbone=Architectures.CNN
)
# builds an actor with a Categorical distribution with a default LSTM with 8 degrees of freedom bound
actor = CategoricalActor(
   out_features=4,
   backbone=Architectures.LSTM
)
# builds an DQN for 8 actions
actor = DQNActor(
   out_features=4,
   backbone=Architectures.MLP
)

The "in-features" will be determined dynamically using lazy modules.

Thoughts?

cc @matteobettini @albertbou92 @BY571 @smorad

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.