[Feature Request] Standardized API for Explicit Multi-Agent Communication Actions

Motivation

Hello,

MARL research seems to move toward communicative agents (learning what, when, and to whom to speak), the library could benefit from a standardized way to define "Communication Actions".

I would like to suggest adding explicit communication between agents as an action for multi-agent environments.
Some environments such as VMAS already include messages in agents' actions but it could be nice to have it in torchRL so that we have a standardized API across different environments.

This would then make it easier to associate rewards and constraints (bandwidth or frequency for example) to communications.

Solution

Creating a specific action could allow existing algorithms to use communications without any modification. For the sender the message would be an action and for the receiver it would be an observation.
We could also make it possible to model asynchronous communication by implementing a Transform that uses a queue to deliver messages to receivers N timesteps after they were sent, thus simulating realistic constraints.

I am fairly new to torchRL so I don't know how these features could be added but I would be happy to provide more details soon if you think this makes sense.

What are your thoughts on this @matteobettini ?

Checklist

• [ x] I have checked that there is no similar issue in the repo (required)