pytest-dev / pytest-dev/pytest-xdist

`TestLoadScope::test_workqueue_ordered_by_input` failing intermittently

Open
#1,248 0 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

Maybe related: #985

I'm doing mass rebuilds of packages in NixOS, with -D_GLIBCXX_DEBUG passed to compilations that link to libstdc++. The latter fact may or may not be relevant.

I am seeing intermittent failures of TestLoadScope::test_workqueue_ordered_by_input in pytest-xdist. It sometimes succeeds, sometimes not, probably depending on things like CPU load.

When it fails, it shows this failure:

________________ TestLoadScope.test_workqueue_ordered_by_input _________________
[gw23] linux -- Python 3.13.7 /nix/store/aj70vbwps8vslhww4p2glmi8jlbr3hp8-python3-3.13.7/bin/python3.13

self = <acceptance_test.TestLoadScope object at 0x7ffff54f0770>
pytester = <Pytester PosixPath('/build/pytest-of-nixbld/pytest-0/popen-gw23/test_workqueue_ordered_by_input0')>

    def test_workqueue_ordered_by_input(self, pytester: pytest.Pytester) -> None:
        test_file = """
            import pytest
            @pytest.mark.parametrize('i', range({}))
            def test(i):
                pass
        """
        pytester.makepyfile(test_a=test_file.format(10), test_b=test_file.format(20))
        result = pytester.runpytest(
            "-n2", "--dist=loadscope", "--no-loadscope-reorder", "-v"
        )
>       assert get_workers_and_test_count_by_prefix(
            "test_a.py::test", result.outlines
        ) == {"gw0": 10}
E       AssertionError: assert {'gw1': 10} == {'gw0': 10}
E
E         Left contains 1 more item:
E         {'gw1': 10}
E         Right contains 1 more item:
E         {'gw0': 10}
E         Use -v to get more diff

/build/pytest_xdist-3.8.0/testing/acceptance_test.py:1268: AssertionError

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with testing/acceptance_test.py, especially TestLoadScope::test_workqueue_ordered_by_input and get_workers_and_test_count_by_prefix. Run the shown pytester scenario with -n2, --dist=loadscope, and --no-loadscope-reorder under varying load to investigate the intermittent worker assignment. Done means the test reliably passes with test_a.py::test assigned to gw0 as expected.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.