facebookresearch / facebookresearch/BenchMARL

Multi-agent environment with early agents termination

Open
#204 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
662
Forks
137
PR merge metrics
No merged PRs in 30d

Description

Hi, I'm trying to work on a simple multi-agent navigation task where agents need to reach a goal position.
I'm giving a little bit of context:
Whenever an agent reaches the desired goal, its done flag becomes true, and whenever an action is passed, I just skip this particular agent and I return a special observation with all -1 and a 0 reward.
I've noticed that in the tensordict generated by a rollout, there are multiple keys {a1,a2,..., done, termination, truncation}.
Since I'm not setting these global done, termination, and truncation keys manually, I was wondering how they are set. In my case, I would like done to be true only when all the agents reach the goal, i.e., {a1: {done==True},a2: {done==True} ...}
Is this already the case?
If not, is it possible to change it? I would like the episode to terminate only when all the agents reach the goal or when the max_timestep is reached.
I'm asking this because I saw that agents are able to reach the goal individually (by looking at my metrics), but they were not even once able to reach the goal at the same time

Contributor guide

Open the contributing guide

Research direction

The report names no files, tests, or entry points. Start by locating the rollout and termination handling for per-agent done, termination, truncation, and max_timestep; verify how global termination is aggregated, and consider the issue complete when the requested all-agents-or-timestep behavior is supported and tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.