pytest-dev / pytest-dev/pytest-xdist

Parametrization-based test scheduling

Open
#511 4 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

I have a test matrix in which the same test suite is run across multiple environments. It's implemented using a fixture which is parametrized with the set of environments, and all tests use it.

I'd like to use xdist, but I couldn't find a test load scheduler that'd fit: All tests running on the same environment must execute sequentially (no specific order is required, of course, but they mustn't run concurrently).

After a quick look under xdist/schedulers, I patched xdist.schedulers.loadfile.LoadFileScheduling._split_scope to split based on the value of the parametrized fixture (which, luckily, appears in the nodeid). This works just fine, but now I'd like to do it without patching xdist in runtime :)

So first of all, if there's any existing way to schedule parametrized tests, I'd be happy to use it instead... And if not, I could use some design help on how my patchy scheduling can be written so it's incorporated into xdist (I'd be happy to create a PR, I think it's quite useful, it was also requested in the past, e.g #324).

I was thinking a --dist=param/nodeid_regex/ argument, then it'd split based on the result.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in xdist/schedulers/loadfile.py, especially LoadFileScheduling._split_scope, and review the existing scheduler behavior and the parametrized fixture pattern described in the issue. Determine how scheduling should group tests by parametrized environment and define the behavior that ensures tests sharing an environment never run concurrently.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.