google-deepmind / google-deepmind/meltingpot

Required test-examples CI check fails for RLlib examples due to missing optional dependencies

Open
#316 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
873
Forks
162
Avg merge
3d 12h
Merged PRs (30d)
29

Description

### Summary

The required `test-examples` CI check currently fails for PRs that touch
RLlib example-related files because the example tests import optional
dependencies (`ray`, `gymnasium`) that are not installed in CI.

As a result, the `test-examples` job fails and **blocks merging**, even when
the PR itself does not modify test logic.

This issue builds on the installation problems discussed in #299, but focuses
specifically on the CI behavior and merge blocking.

---

### Observed CI behavior

- `test-examples` is a required check for merging PRs.
- CI does not install `examples/requirements.in`.
- RLlib example tests under `examples/rllib/` import Ray and Gymnasium at
module import time.
- During CI:
- tests fail with `ModuleNotFoundError`, or
- the test job hangs or errors during collection.
- This causes `test-examples` to fail and **prevents PRs from being merged**.

Example failure from CI:

```
ERROR examples/rllib/self_play_train_test.py
ModuleNotFoundError: No module named 'ray'
```

---

### Expected behavior

One of the following should be true:

- Example tests explicitly skip when optional dependencies are not installed, or
- CI installs the example dependencies before running `test-examples`, or
- Example tests are excluded from required CI checks.

In all cases, example-related tests should not cause mandatory CI failures
when optional dependencies are intentionally absent.

---

### Additional context

PR #315 pins the RLlib example dependencies to a known-working set, addressing
runtime incompatibilities reported in #299. However, the PR is currently
blocked because `test-examples` fails due to the behavior described above.

This issue is intended to document the CI failure and discuss the preferred
long-term solution.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the required `test-examples` check and inspect `examples/rllib/self_play_train_test.py`, `examples/requirements.in`, and the CI configuration. Compare the available handling options, including the dependency pinning context in PR #315. Done means RLlib example-related changes no longer block merging when optional dependencies are absent, while intended example tests remain covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.