pytest-dev / pytest-dev/pytest-xdist

pytestmark with loadgroup not working

Open
#1,295 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

According to https://pytest-xdist.readthedocs.io/en/stable/distribution.html

--dist loadgroup: Tests are grouped by the xdist_group mark. Groups are distributed to available workers as whole units. This guarantees that all tests with same xdist_group name run in the same worker. If a test has multiple groups, they will be joined together into a new group, the order of the marks doesn’t matter. This works along with marks from fixtures and from the pytestmark global variable.

I'm seeing issues when attempting to use --dist loadgroup with pytestmark

Have a simple reproduction

.
├── bar
│  ├── __init__.py
│  └── tests
│     ├── conftest.py
│     └── test_bar.py
├── conftest.py
├── foo
│  ├── __init__.py
│  └── tests
│     ├── conftest.py
│     └── test_foo.py
├── main.py
├── pyproject.toml
├── README.md
└── uv.lock

The root conftest.py

import os
import logging

def pytest_configure(config):
    worker_id = os.environ.get("PYTEST_XDIST_WORKER")
    if worker_id is not None:
        logging.basicConfig(
            format=config.getini("log_file_format"),
            filename=f"tests_{worker_id}.log",
            level=config.getini("log_file_level"),
        )

foo/tests/conftest.py looks like

import logging

import pytest
from flask import Flask

pytestmark = pytest.mark.xdist_group(name="foo")

logger = logging.getLogger(__name__)


@pytest.fixture(scope="package")
def app():
    logger.warning("foo setup")
    yield Flask("test")
    logger.warning("foo teardown")

bar/tests/conftest.py looks like

import logging

import pytest
from flask import Flask

pytestmark = pytest.mark.xdist_group(name="foo")

logger = logging.getLogger(__name__)


@pytest.fixture(scope="package")
def app():
    logger.warning("foo setup")
    yield Flask("test")
    logger.warning("foo teardown")

test_foo.py and test_bar.py contain x2 empty tests respectively


Bug

When I run

uv run pytest -n 2 --dist loadgroup

I get the following

====================================================== test session starts =======================================================
platform darwin -- Python 3.13.3, pytest-9.0.2, pluggy-1.6.0 -- /Users/jack/code/hehe/.venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/jack/code/hehe
configfile: pyproject.toml
plugins: xdist-3.8.0
2 workers [4 items]
scheduling tests via LoadGroupScheduling

bar/tests/test_bar.py::test_bar_b
bar/tests/test_bar.py::test_bar_a
[gw1] [ 25%] PASSED bar/tests/test_bar.py::test_bar_b
[gw0] [ 50%] PASSED bar/tests/test_bar.py::test_bar_a
foo/tests/test_foo.py::test_foo_b
foo/tests/test_foo.py::test_foo_a
[gw0] [ 75%] PASSED foo/tests/test_foo.py::test_foo_a
[gw1] [100%] PASSED foo/tests/test_foo.py::test_foo_b

======================================================= 4 passed in 0.28s ========================================================

it looks like gw0 and gw1 and running tests in both foo and bar.

This is more obvious looking in the logs

───────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: tests_gw0.log
───────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ bar setup
   2   │ foo setup
   3   │ foo teardown
   4   │ bar teardown
───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
───────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: tests_gw1.log
───────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ bar setup
   2   │ foo setup
   3   │ foo teardown
   4   │ bar teardown
───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Given I have x2 workers and x2 groups, I'd expect gw0 to run foo completley and gw1 to run bar completely with no overlap.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the supplied reproduction with uv run pytest -n 2 --dist loadgroup and inspect the root, foo/tests/conftest.py, bar/tests/conftest.py, and their test files. Start from the reported LoadGroupScheduling behavior and verify how pytestmark is applied. Done means tests sharing each group run on one worker without both package fixtures being set up on both workers.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.