[BUG] NoisyLinear uses self.training to turn on/off noise

Describe the bug

https://github.com/pytorch/rl/blob/2fd46326e2eb8d155258feed81dbce1574f527f0/torchrl/modules/models/exploration.py#L131

In PyTorch, module.training is used to differentiate training and inference. For example, Normalization layers use training attribute because they should behave differently when training and inference. However, NoisyLinear is not the case, NoisyLinear should stay active both during training and during inference (actually this is the whole point of NoisyLinear, it "learns" noisy parameters during training, and uses noise for exploration during inference). The only time NoisyLinear should be turned off is during policy evaluation/testing. (evaluation is not the same as inference in RL, evaluation should always be inference, while inference can be either evaluation or exploration).

In the current implementation, to correctly allow the policy to explore, module.training must be turned on even for inference during exploration, however, turning on training mode will unavoidably affect normalization layers, dropout, etc. Although Dropout is not widely used in RL yet, normalization layers are sometimes being used in recent years (especially in some works that try to use Transformers in RL). I personally think using training attribute, in this case, is a bug.

To Reproduce

Expected behavior

NoisyLinear should be activated during exploration/training, and deactivated during evaluation/testing. Other pytorch modules like normalization should not be impacted by the behavior of NoisyLinear.

Screenshots

System info

Additional context

Reason and Possible fixes

possible fix: add an additional function or attribute specifically for NoisyLinear to turn on and off noise (just like how its reset_noise resets noise)

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have read the documentation (required)
• I have provided a minimal working example to reproduce the bug (required)