feat(composer): agent specific observations
- Dominant language
- Jupyter Notebook
- Stars
- 3.2k
- Forks
- 349
- PR merge metrics
- No merged PRs in 30d
Description
Hey 👋
I am working in my fork on supporting [agent specific observations](https://github.com/google/brax/blob/8a3ea6065866f687cf657d4fc5c91d002a75f72f/brax/experimental/composer/agent_utils.py#L36) in the composer sub-package. Additionally I need this feature to support observation masking that is dependent on the current state of the environment.
Best approach I can see:
1. Each agent component already accepts `observers`
a) Similar to how [`reward_fns`](https://github.com/google/brax/blob/b3e75f9f0a66c19a3d76f232218b5029f116f3dd/brax/experimental/composer/composer.py#L45) is done
b) This could support dynamic masking
2. Introduce two methods to flatten and unflatten the observation array so that each agent can be given it's specific observation.
3. Pass this sort of information inside the `edge` parameter to `Composer`:
a) Could be part of its responsibility[ "automatically create necessary edge information among 2+ components"](https://github.com/google/brax/blob/b3e75f9f0a66c19a3d76f232218b5029f116f3dd/brax/experimental/composer/composer.py#L47)
b) But maybe edge information is only meant to care about [pairs of components](https://github.com/google/brax/blob/b3e75f9f0a66c19a3d76f232218b5029f116f3dd/brax/experimental/composer/composer.py#L207)
Let me know if this sounds cool with you guys, but otherwise I am going to make a start on this feature 😀
Contributor guide
Assessment
This issue has not been assessed yet.