pytorch / pytorch/rl

[Feature Request] Information of RNNs expected inputs and outputs difficult to access when part of larger architectures

Open
#1,771 2 comments 1 reaction 1 assignee View on GitHub

@vmoens is already working on this.

Since Jan 4, 2024.

enhancement
Dominant language
Python
Stars
3.6k
Forks
487
Avg merge
1d 1h
Merged PRs (30d)
207

Description

Motivation

When RNN’s are used in isolation, creating a TensorDictPrimer Transform for the environment to populate the TensorDicts with the expected tensors is pretty straightforward:

from torchrl.modules import GRUModule

gru_module = GRUModule(
    input_size=10,
    hidden_size=10,
    num_layers=1,
    in_keys=["input", "recurrent_state", "is_init"],
    out_keys=["output", ("next", "recurrent_state")],
)

transform = gru_module.make_tensordict_primer()

However, when RNN’s are part of a larger architecture, this can become tricky. e.g.

from torchrl.modules import GRUModule, MLP
from tensordict.nn import TensorDictModule, TensorDictSequential

gru_module = GRUModule(
    input_size=10,
    hidden_size=10,
    num_layers=1,
    in_keys=["input", "recurrent_state", "is_init"],
    out_keys=["features", ("next", "recurrent_state")],
)
head = TensorDictModule(
    MLP(
        in_features=10,
        out_features=10,
        num_cells=[],
    ),
    in_keys=["features"],
    out_keys=["output"],
)
model = TensorDictSequential(gru_module, head)

In case you know the architecture, it is still possible to do:

transform = model[0].make_tensordict_primer()

But this is not ideal. Besides, beyond creating the transform automatically, maybe the user is interested in knowing the required shapes and other information of the model inputs, which now has the RNN inputs and their own inputs.

Solution

A solution would be to make possible to access all the information about the model expected inputs and outputs from some model specs.

Maybe it should not be required to define specs during the creation of the model, but optionally adding input specs would facilitate creating the primer transform in these cases.

Checklist

  • I have checked that there is no similar issue in the repo (required)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.