Is per-agent advantage normalization intentionally left out?

Hi, I noticed in TorchRL, there's `normalize_advantage_exclude_dims` which normalize advantage without mixing agent dimention; however, BenchMARL doesn't seem to have adopted it. I also notice an imbalance `loss_entropy` and `loss_objective` when training with MAPPO and IPPO.

Is this implementation detail deliberate?

Start by tracing how BenchMARL normalizes advantages for MAPPO and IPPO, then compare that behavior with TorchRL's normalize_advantage_exclude_dims. Inspect how loss_entropy and loss_objective are computed and whether their imbalance is reproducible. Done means establishing whether the omission is deliberate and documenting or correcting the behavior based on the result.

Written by the indexing model from the issue text.