pytest-dev / pytest-dev/pytest-xdist
Overhead of pytest-xdist is often greater than time saved for parallelization
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 287
- Avg merge
- 9h 30m
- Merged PRs (30d)
- 2
Description
(I'm pretty sure this is a known issue, but I didn't see an open issue for it, and wanted there to be a canonical place to track it)
Example:
(cryptography) ~/p/cryptography (master) $ py.test tests/test_x509*.py
====================================================== test session starts =======================================================
platform darwin -- Python 2.7.10[pypy-4.0.1-final], pytest-2.8.7, py-1.4.31, pluggy-0.3.1
rootdir: /Users/alex_gaynor/projects/cryptography, inifile: tox.ini
plugins: hypothesis-2.0.0, xdist-1.14
collected 492 items
tests/test_x509.py .........................................................................................................................................................................................
tests/test_x509_crlbuilder.py ............................
tests/test_x509_ext.py .......................................................................................................................................................................................................................................................................
tests/test_x509_revokedcertbuilder.py ................
=================================================== 492 passed in 3.60 seconds ===================================================
(cryptography) ~/p/cryptography (master) $ py.test -n 2 tests/test_x509*.py
====================================================== test session starts =======================================================
platform darwin -- Python 2.7.10[pypy-4.0.1-final], pytest-2.8.7, py-1.4.31, pluggy-0.3.1
rootdir: /Users/alex_gaynor/projects/cryptography, inifile: tox.ini
plugins: hypothesis-2.0.0, xdist-1.14
gw0 [492] / gw1 [492]
scheduling tests via LoadScheduling
............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
=================================================== 492 passed in 5.00 seconds ===================================================
(cryptography) ~/p/cryptography (master) $ py.test -n 2 tests/test_x509*.py
====================================================== test session starts =======================================================
platform darwin -- Python 2.7.10[pypy-4.0.1-final], pytest-2.8.7, py-1.4.31, pluggy-0.3.1
rootdir: /Users/alex_gaynor/projects/cryptography, inifile: tox.ini
plugins: hypothesis-2.0.0, xdist-1.14
gw0 [492] / gw1 [492]
scheduling tests via LoadScheduling
............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
=================================================== 492 passed in 4.62 seconds ===================================================
(cryptography) ~/p/cryptography (master) $ py.test -n 4 tests/test_x509*.py
====================================================== test session starts =======================================================
platform darwin -- Python 2.7.10[pypy-4.0.1-final], pytest-2.8.7, py-1.4.31, pluggy-0.3.1
rootdir: /Users/alex_gaynor/projects/cryptography, inifile: tox.ini
plugins: hypothesis-2.0.0, xdist-1.14
gw0 [492] / gw1 [492] / gw2 [492] / gw3 [492]
scheduling tests via LoadScheduling
............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
=================================================== 492 passed in 6.23 seconds ===================================================
(cryptography) ~/p/cryptography (master) $ py.test -n 4 tests/test_x509*.py
====================================================== test session starts =======================================================
platform darwin -- Python 2.7.10[pypy-4.0.1-final], pytest-2.8.7, py-1.4.31, pluggy-0.3.1
rootdir: /Users/alex_gaynor/projects/cryptography, inifile: tox.ini
plugins: hypothesis-2.0.0, xdist-1.14
gw0 [492] / gw1 [492] / gw2 [492] / gw3 [492]
scheduling tests via LoadScheduling
............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
=================================================== 492 passed in 6.73 seconds ===================================================
(cryptography) ~/p/cryptography (master) $ py.test -n 4 tests/test_x509*.py
====================================================== test session starts =======================================================
platform darwin -- Python 2.7.10[pypy-4.0.1-final], pytest-2.8.7, py-1.4.31, pluggy-0.3.1
rootdir: /Users/alex_gaynor/projects/cryptography, inifile: tox.ini
plugins: hypothesis-2.0.0, xdist-1.14
gw0 [492] / gw1 [492] / gw2 [492] / gw3 [492]
scheduling tests via LoadScheduling
............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
=================================================== 492 passed in 6.53 seconds ===================================================
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue provides benchmark commands for pytest and pytest-xdist using tests/test_x509*.py, but names no source file or failing test. Start by reproducing the serial and -n 2/-n 4 runs, then inspect pytest-xdist's parallel-test startup and scheduling behavior; completion criteria are not defined beyond addressing the observed overhead.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100