pytorch / pytorch/rl

[Bug/Question] Target workflow for LSTM Modules?

Open
#2,104 0 comments 0 reactions 1 assignee View on GitHub

@vmoens is already working on this.

Since Apr 23, 2024.

bug
Dominant language
Python
Stars
3.6k
Forks
484
Avg merge
1d 1h
Merged PRs (30d)
207

Description

Describe the question

In the test suites, LSTMModules are created with inputs and outputs before the environment is created. As such, the primer is added to the environment before parallelization. In "sample code", environments are created and specs are used to design/shape inputs and outputs to modules. With the environment first workflow the LSTMModule primer ends up being added to the finished parallelenv, but the create primer method ignores any batch dims, causing a runtime error when the transform_observation_spec is run, because the assumption is that the specs are correct.

To Reproduce

Hack one of the SOTA implementations to create an LSTM module. Add a primer at the end.

The question is whether the create_primer should be updated (maybe optionally take a batch spec), or the Primer logic should not assume that specs are correct, or that the workflow should be as in the tests. The way it is causes a bit of a surprise, though.

cc @albertbou92

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.