facebookresearch / facebookresearch/BenchMARL

Environment transform weights not loaded at eval time

Open
#254 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
662
Forks
137
PR merge metrics
No merged PRs in 30d

Description

I'm willing to contribute a PR to fix this, but I've noticed that eval time, learnable environment transforms don't have their weights loaded. For example, VecNorm is loaded, but the weights are not. VecNorm is also loaded in an unfrozen state. To me, this seems strange as at eval you'd want to have the observations normalized in the same way that was seen in training. Was there a reason for this?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing how environment transforms, especially VecNorm, are loaded for evaluation and how their weights and frozen state are handled. Compare this with the training-time transform setup and reproduce evaluation from a trained checkpoint. Done means the learned transform weights are restored and evaluation uses the same observation normalization seen during training.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.