google / google/brax

feat(composer): agent specific observations

Open
#257 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
3.2k
Forks
349
PR merge metrics
No merged PRs in 30d

Description

Hey 👋

I am working in my fork on supporting [agent specific observations](https://github.com/google/brax/blob/8a3ea6065866f687cf657d4fc5c91d002a75f72f/brax/experimental/composer/agent_utils.py#L36) in the composer sub-package. Additionally I need this feature to support observation masking that is dependent on the current state of the environment.

Best approach I can see:
1. Each agent component already accepts `observers`
a) Similar to how [`reward_fns`](https://github.com/google/brax/blob/b3e75f9f0a66c19a3d76f232218b5029f116f3dd/brax/experimental/composer/composer.py#L45) is done
b) This could support dynamic masking
2. Introduce two methods to flatten and unflatten the observation array so that each agent can be given it's specific observation.
3. Pass this sort of information inside the `edge` parameter to `Composer`:
a) Could be part of its responsibility[ "automatically create necessary edge information among 2+ components"](https://github.com/google/brax/blob/b3e75f9f0a66c19a3d76f232218b5029f116f3dd/brax/experimental/composer/composer.py#L47)
b) But maybe edge information is only meant to care about [pairs of components](https://github.com/google/brax/blob/b3e75f9f0a66c19a3d76f232218b5029f116f3dd/brax/experimental/composer/composer.py#L207)

Let me know if this sounds cool with you guys, but otherwise I am going to make a start on this feature 😀

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.