pytest-dev / pytest-dev/pytest-xdist
test item scheduling does not schedule N items to N workers
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 287
- Avg merge
- 9h 30m
- Merged PRs (30d)
- 2
Description
I may just be misunderstanding how xdist works, but I can't seem to get the number of workers I requested to all accept tasks in parallel:
$ py.test -v -n 8
================================ test session starts =================================
platform darwin -- Python 3.5.1, pytest-2.8.5, py-1.4.31, pluggy-0.3.1 -- .../pytest-xdist-testcase/venv/bin/python3
cachedir: .cache
rootdir: .../pytest-xdist-testcase, inifile:
plugins: xdist-1.13.1
[gw0] darwin Python 3.5.1 cwd: .../pytest-xdist-testcase
[gw1] darwin Python 3.5.1 cwd: .../pytest-xdist-testcase
[gw2] darwin Python 3.5.1 cwd: .../pytest-xdist-testcase
[gw3] darwin Python 3.5.1 cwd: .../pytest-xdist-testcase
[gw4] darwin Python 3.5.1 cwd: .../pytest-xdist-testcase
[gw5] darwin Python 3.5.1 cwd: .../pytest-xdist-testcase
[gw6] darwin Python 3.5.1 cwd: .../pytest-xdist-testcase
[gw7] darwin Python 3.5.1 cwd: .../pytest-xdist-testcase
[gw0] Python 3.5.1 (default, Dec 7 2015, 21:59:10) -- [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)]
[gw1] Python 3.5.1 (default, Dec 7 2015, 21:59:10) -- [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)]
[gw2] Python 3.5.1 (default, Dec 7 2015, 21:59:10) -- [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)]
[gw3] Python 3.5.1 (default, Dec 7 2015, 21:59:10) -- [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)]
[gw4] Python 3.5.1 (default, Dec 7 2015, 21:59:10) -- [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)]
[gw5] Python 3.5.1 (default, Dec 7 2015, 21:59:10) -- [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)]
[gw6] Python 3.5.1 (default, Dec 7 2015, 21:59:10) -- [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)]
[gw7] Python 3.5.1 (default, Dec 7 2015, 21:59:10) -- [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)]
gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] / gw6 [8] / gw7 [8]
scheduling tests via LoadScheduling
test_tests.py::test_1[0]
test_tests.py::test_1[2]
test_tests.py::test_2[0]
test_tests.py::test_2[2]
[gw0] PASSED test_tests.py::test_1[2]
[gw7] PASSED test_tests.py::test_1[0]
[gw2] PASSED test_tests.py::test_2[0]
[gw4] PASSED test_tests.py::test_2[2]
test_tests.py::test_2[1]
test_tests.py::test_1[3]
test_tests.py::test_1[1]
test_tests.py::test_2[3]
[gw7] PASSED test_tests.py::test_1[1]
[gw0] PASSED test_tests.py::test_1[3]
[gw4] PASSED test_tests.py::test_2[3]
[gw2] PASSED test_tests.py::test_2[1]
============================= 8 passed in 64.12 seconds ==============================
Watching the output as it comes, it looks like the second block of 4 tests doesn't start running until the first block of 4 completes.
Am I misunderstanding something? Why don't all 8 workers start running tests at the same time? Why doesn't gw1, for example, get any tests?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the report with py.test -v -n 8 and the eight parametrized tests shown in the issue. Start by tracing the LoadScheduling behavior and worker assignments; done means explaining or correcting why only four tests run initially and why gw1 receives no test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100