google-deepmind / google-deepmind/meltingpot

Coop-mining environment bug

Open
#293 3 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
873
Forks
162
Avg merge
3d 12h
Merged PRs (30d)
29

Description

I have been using the cooperative mining environment and training agents to perform the task. I encountered two potential issues related to multiple agents shooting at gold:

1. Limited Shot Registration:

When more than two agents attempt to shoot at gold simultaneously, only the first two agents receive rewards. Although multiple agents can execute the shooting action at the same time, only two shots are registered and counted as a mining event ([registering mining event in lua code).](https://github.com/google-deepmind/meltingpot/blob/b56ad1a5a53cc63c1e5c84b504562492e03ebddd/meltingpot/lua/levels/coop_mining/components.lua#L123)

2. Gold Persistence and Excess Rewards:

When more than two agents shoot at the gold, the gold does not disappear immediately, even though two agents receive rewards. This results in agents seemingly receiving double or triple rewards for mining the same gold.

I have recorded a video with three agents, where I display each agent's actions and rewards at each time step. In the video, action index 7 corresponds to the shooting action. Reward for iron mining grants 0.9, and gold mining grants 5.9 (after applying a -0.1 penalty per shooting action).

Video Link: https://drive.google.com/file/d/1PgsrAodoNAD5wrQrW6d9IjnGyIXkWDm7/view?usp=sharing
The issues occur frequently in the last 30 seconds of the video.

I appreciate any insights on why this might happen, thank you!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.