[Feature Request] Prebuilt models
Open
@vmoens is already working on this.
Since Jul 28, 2023.
enhancement
- Dominant language
- Python
- Stars
- 3.6k
- Forks
- 487
- Avg merge
- 23h 50m
- Merged PRs (30d)
- 209
Description
Motivation
It can be unnecessary tedious to write actors (and value) models.
A possible solution would be to have pre-built TensorDictSequence subclasses.
Here are a few examples:
from torchrl.modules import Architectures
# builds an actor with a TanhNormal distribution with [-1, 1] bound
actor = TanhActor(
out_features=4,
backbone=Architectures.MLP
)
# builds an actor with a Categorical distribution with 8 degrees of freedom bound
actor = CategoricalActor(
out_features=4,
backbone=Architectures.MLP
)
# builds an actor with a Categorical distribution for pixels with 8 degrees of freedom bound
actor = CategoricalActor(
out_features=4,
backbone=Architectures.CNN
)
# builds an actor with a Categorical distribution with a default LSTM with 8 degrees of freedom bound
actor = CategoricalActor(
out_features=4,
backbone=Architectures.LSTM
)
# builds an DQN for 8 actions
actor = DQNActor(
out_features=4,
backbone=Architectures.MLP
)
The "in-features" will be determined dynamically using lazy modules.
Thoughts?
cc @matteobettini @albertbou92 @BY571 @smorad
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.