facebookresearch / facebookresearch/ZeroSumEval

Add Diplomacy 🗺️

Open
#7 0 comments 0 reactions 0 assignees View on GitHub
new game
Dominant language
Python
Stars
35
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Implement the game of Diplomacy as a ZeroSumEval-compatible game. This game requires multi-agent negotiation, long-term planning, and coalition-building. A great fit for evaluating strategic communication and intent modeling in LLMs. Ideal for 3+ players. Includes challenges around turn structure and map-based game state.

This game is likely to be non-trivial to implement and play due to the large amount of information that needs to be encoded in text to be fed to an LLM. This could be a good candidate for multimodal models, where we could pass the current state of the board as an image instead.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.