pytest-dev / pytest-dev/pytest-xdist

loadscope on parametrized classes not going parallel

Open
#759 5 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

I am parametrizing a test class and scheduling tests with --dist loadscope. I was expecting the effect that every parameter combination will be treated as a separate class by pytest. Then, those parametrized runs should be able to go to different pytest processes when using loadscope. However, I am not seeing that behavior. All runs stay on the same process always. There doesn't seem to be any different in scheduling behavior when using loadscope between parametrizing a test method versus parametrizing a test class.

What is the expected behavior with loadscope and parametrized classes? And if they are supposed to run in parallel, is this a bug?

(lux-D-BxQKsC-py3.8) ➜  test cat testp.py
import pytest

@pytest.mark.parametrize("paramvalue", ["p1", "p2"])
class TestClass:
  def test_method_1(self, paramvalue):
    assert True

  def test_method_2(self, paramvalue):
    assert True
(lux-D-BxQKsC-py3.8) ➜  test pytest -n auto --dist loadscope -vvv testp.py
==================================================== test session starts ====================================================
platform darwin -- Python 3.8.12, pytest-6.2.4, py-1.11.0, pluggy-0.13.1 -- /Users/manub/Library/Caches/pypoetry/virtualenvs/lux-D-BxQKsC-py3.8/bin/python
cachedir: .pytest_cache
rootdir: /Users/manub/tmp/test
plugins: xdist-2.5.0, forked-1.4.0, rerunfailures-10.2, mock-3.7.0, dotenv-0.5.2, profiling-1.7.0
[gw0] darwin Python 3.8.12 cwd: /Users/manub/tmp/test
[gw1] darwin Python 3.8.12 cwd: /Users/manub/tmp/test
[gw2] darwin Python 3.8.12 cwd: /Users/manub/tmp/test
[gw3] darwin Python 3.8.12 cwd: /Users/manub/tmp/test
[gw4] darwin Python 3.8.12 cwd: /Users/manub/tmp/test
[gw5] darwin Python 3.8.12 cwd: /Users/manub/tmp/test
[gw0] Python 3.8.12 (default, Nov 29 2021, 23:59:10)  -- [Clang 11.0.3 (clang-1103.0.32.62)]
[gw1] Python 3.8.12 (default, Nov 29 2021, 23:59:10)  -- [Clang 11.0.3 (clang-1103.0.32.62)]
[gw2] Python 3.8.12 (default, Nov 29 2021, 23:59:10)  -- [Clang 11.0.3 (clang-1103.0.32.62)]
[gw3] Python 3.8.12 (default, Nov 29 2021, 23:59:10)  -- [Clang 11.0.3 (clang-1103.0.32.62)]
[gw4] Python 3.8.12 (default, Nov 29 2021, 23:59:10)  -- [Clang 11.0.3 (clang-1103.0.32.62)]
[gw5] Python 3.8.12 (default, Nov 29 2021, 23:59:10)  -- [Clang 11.0.3 (clang-1103.0.32.62)]
gw0 [4] / gw1 [4] / gw2 [4] / gw3 [4] / gw4 [4] / gw5 [4]
scheduling tests via LoadScopeScheduling

testp.py::TestClass::test_method_1[p1] 
[gw0] [ 25%] PASSED testp.py::TestClass::test_method_1[p1] 
testp.py::TestClass::test_method_1[p2] 
[gw0] [ 50%] PASSED testp.py::TestClass::test_method_1[p2] 
testp.py::TestClass::test_method_2[p1] 
[gw0] [ 75%] PASSED testp.py::TestClass::test_method_2[p1] 
testp.py::TestClass::test_method_2[p2] 
[gw0] [100%] PASSED testp.py::TestClass::test_method_2[p2] 

===================================================== 4 passed in 0.86s =====================================================
(lux-D-BxQKsC-py3.8) ➜  test 

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the behavior from testp.py with pytest -n auto --dist loadscope -vvv, then read the LoadScopeScheduling entry point and its scheduling behavior. Confirm whether parametrized class runs should form separate scheduling groups; done means the behavior is documented or corrected and covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.