ReactionMechanismGenerator / ReactionMechanismGenerator/ARC
Flaky: test_initialize_output_dict fails intermittently under xdist (order-dependent)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 51
- Forks
- 25
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 15
Description
arc/scheduler_test.py::TestScheduler::test_initialize_output_dict fails intermittently in CI, on main and on PRs that do not touch the scheduler at all.
FAILED arc/scheduler_test.py::TestScheduler::test_initialize_output_dict - AssertionError: False is not true
The assertion is self.assertTrue(self.sched1._does_output_dict_contain_info()) (arc/scheduler_test.py:350).
Evidence that it is order-dependent, not a real defect
- On
main: the nightly run of 2026-08-10 (4a63d1aa) failed on this test — https://github.com/ReactionMechanismGenerator/ARC/actions/runs/31349161224 — with green nightlies on 08-11, 08-12 and 08-13 either side of it, and no relevant change in between. - On an unrelated PR: #981 (
yaml_bool_smiles_fix) failed the same way — https://github.com/ReactionMechanismGenerator/ARC/actions/runs/31736868393/job/94570512325 — reported bygw4at 1% of the run. That branch only touchesarc/common.pyandarc/species/species.py(plus their tests) and never goes near the scheduler. - Locally: running the test on its own reproduces the failure on a checkout of
main's content:
so the test passes only when earlier tests in the class have run first. UnderHOME=$(mktemp -d) python -m pytest arc/scheduler_test.py -q -n0 -k test_initialize_output_dict → 1 failed, 42 deselectedpytest-xdistthe shard that receives it may not have run them, which is why it surfaces sporadically rather than always.
Suspected cause
self.sched1 is built in setUpClass, and _does_output_dict_contain_info() reads state that other tests in the class populate. A likely contributor is global mutation of default_job_types: initialize_job_types() aliases and dels the module-level global, so a test that calls it changes what a later Scheduler sees. PR #926 carries a commit "Stop initialize_job_types from mutating the global default_job_types", which may remove the cause; worth re-checking this test's stability once that lands.
Suggested fix
Make the test self-sufficient rather than dependent on sibling tests — populate the output dict it asserts on inside the test (or in setUp), so it passes under -p no:randomly, in isolation, and on any xdist shard.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with arc/scheduler_test.py::TestScheduler::test_initialize_output_dict and the assertion at line 350. Run the named test in isolation with the provided pytest command, then inspect setUpClass and the state used by _does_output_dict_contain_info(). It is done when the test passes alone, with -p no:randomly, and on an xdist shard without relying on sibling tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100