google-deepmind / google-deepmind/meltingpot
Required test-examples CI check fails for RLlib examples due to missing optional dependencies
- Dominant language
- Python
- Stars
- 873
- Forks
- 162
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 29
Description
### Summary
The required `test-examples` CI check currently fails for PRs that touch
RLlib example-related files because the example tests import optional
dependencies (`ray`, `gymnasium`) that are not installed in CI.
As a result, the `test-examples` job fails and **blocks merging**, even when
the PR itself does not modify test logic.
This issue builds on the installation problems discussed in #299, but focuses
specifically on the CI behavior and merge blocking.
---
### Observed CI behavior
- `test-examples` is a required check for merging PRs.
- CI does not install `examples/requirements.in`.
- RLlib example tests under `examples/rllib/` import Ray and Gymnasium at
module import time.
- During CI:
- tests fail with `ModuleNotFoundError`, or
- the test job hangs or errors during collection.
- This causes `test-examples` to fail and **prevents PRs from being merged**.
Example failure from CI:
```
ERROR examples/rllib/self_play_train_test.py
ModuleNotFoundError: No module named 'ray'
```
---
### Expected behavior
One of the following should be true:
- Example tests explicitly skip when optional dependencies are not installed, or
- CI installs the example dependencies before running `test-examples`, or
- Example tests are excluded from required CI checks.
In all cases, example-related tests should not cause mandatory CI failures
when optional dependencies are intentionally absent.
---
### Additional context
PR #315 pins the RLlib example dependencies to a known-working set, addressing
runtime incompatibilities reported in #299. However, the PR is currently
blocked because `test-examples` fails due to the behavior described above.
This issue is intended to document the CI failure and discuss the preferred
long-term solution.
Contributor guide
Research direction
Start by reproducing the required `test-examples` check and inspect `examples/rllib/self_play_train_test.py`, `examples/requirements.in`, and the CI configuration. Compare the available handling options, including the dependency pinning context in PR #315. Done means RLlib example-related changes no longer block merging when optional dependencies are absent, while intended example tests remain covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100