pytest-dev / pytest-dev/pytest-xdist

Support --boxed on Windows

Open
#270 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

I am developing a C/C++ module for Python (3.6 64-bit) under Windows, and would like to use the pytest + xdist framework to write tests for it. Each test should run in its own process, in case there is a crash in one of the tests. xdist supports this with the --boxed option, but unfortunately this does not work under Windows (as xdist uses os.fork provided by _process/forkedfunc.py to spawn subprocesses).

I tried extending this using the multiprocessing package, but without success so far. It requires serializing the pytest.Item object (multiprocessing uses pickle to exchange functions and objects between processes), which seems far from simple given its complexity. My idea is to convert pytest.Item to a serializable object (e.g. SerializableItem), which can be used to generate a proper pytest.Item in the subprocess. I got as far as reaching the Pytest runner with the generated Item, but I am stuck with generating pytest.Item._request.
Has anyone tried a similar approach? Maybe it is the wrong way to go given the complexity of the Item class.

I also tried to serialize pytest.Item using dill, which went much further than pickle, but the whole object could not be serialized (some part of the pytest.Config as far as I can tell).

Any help with this would be very welcome :-)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with _process/forkedfunc.py and the existing --boxed path, then review how pytest.Item and its _request are handled across processes. The issue’s proposed multiprocessing and serialization approaches are exploratory; done would mean --boxed supports running each test in its own process on Windows.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.