ReactionMechanismGenerator / ReactionMechanismGenerator/ARC

Flaky: test_initialize_output_dict fails intermittently under xdist (order-dependent)

Open Beginner friendly
#984 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
51
Forks
25
Avg merge
4d 5h
Merged PRs (30d)
15

Description

arc/scheduler_test.py::TestScheduler::test_initialize_output_dict fails intermittently in CI, on main and on PRs that do not touch the scheduler at all.

FAILED arc/scheduler_test.py::TestScheduler::test_initialize_output_dict - AssertionError: False is not true

The assertion is self.assertTrue(self.sched1._does_output_dict_contain_info()) (arc/scheduler_test.py:350).

Evidence that it is order-dependent, not a real defect
  • On main: the nightly run of 2026-08-10 (4a63d1aa) failed on this test — https://github.com/ReactionMechanismGenerator/ARC/actions/runs/31349161224 — with green nightlies on 08-11, 08-12 and 08-13 either side of it, and no relevant change in between.
  • On an unrelated PR: #981 (yaml_bool_smiles_fix) failed the same way — https://github.com/ReactionMechanismGenerator/ARC/actions/runs/31736868393/job/94570512325 — reported by gw4 at 1% of the run. That branch only touches arc/common.py and arc/species/species.py (plus their tests) and never goes near the scheduler.
  • Locally: running the test on its own reproduces the failure on a checkout of main's content:
    HOME=$(mktemp -d) python -m pytest arc/scheduler_test.py -q -n0 -k test_initialize_output_dict
    → 1 failed, 42 deselected
    
    so the test passes only when earlier tests in the class have run first. Under pytest-xdist the shard that receives it may not have run them, which is why it surfaces sporadically rather than always.
Suspected cause

self.sched1 is built in setUpClass, and _does_output_dict_contain_info() reads state that other tests in the class populate. A likely contributor is global mutation of default_job_types: initialize_job_types() aliases and dels the module-level global, so a test that calls it changes what a later Scheduler sees. PR #926 carries a commit "Stop initialize_job_types from mutating the global default_job_types", which may remove the cause; worth re-checking this test's stability once that lands.

Suggested fix

Make the test self-sufficient rather than dependent on sibling tests — populate the output dict it asserts on inside the test (or in setUp), so it passes under -p no:randomly, in isolation, and on any xdist shard.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with arc/scheduler_test.py::TestScheduler::test_initialize_output_dict and the assertion at line 350. Run the named test in isolation with the provided pytest command, then inspect setUpClass and the state used by _does_output_dict_contain_info(). It is done when the test passes alone, with -p no:randomly, and on an xdist shard without relying on sibling tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.