pytest-dev / pytest-dev/pytest-xdist

`pytest_xdist_auto_num_workers` function cannot be conditionally defined

Open
#1,102 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

the problem

i'm trying to create a pytest_xdist_auto_num_workers hook that should only be registered if the user runs pytest with the xdist plugin enabled, otherwise it will crash due to the hook name being unknown:

according to the pytest docs, i can accomplish it like so:

class DeferPlugin:
    @hookimpl(wrapper=True)
    def pytest_xdist_auto_num_workers(config: pytest.Config) -> Generator[None, int, int]:
        return min((yield), len(config.option.file_or_dir))


def pytest_configure(config):
    if config.pluginmanager.hasplugin("xdist"):
        config.pluginmanager.register(DeferPlugin())

however it doesn't seem to work. the DeferPlugin gets registered but the pytest_xdist_auto_num_workers hook never gets called. i believe this is because pytest_xdist_auto_num_workers gets called before pytest_configure.

attempted workarounds

PYTEST_XDIST_AUTO_NUM_WORKERS environment variable

i considered using the PYTEST_XDIST_AUTO_NUM_WORKERS environment variable, but that won't work in this case because the logic i want to use to determine the new value needs to be based on the default value, which the environment variable does not contain.

attempting to import xdist before registering the hook

i tried doing this:

try:
    import xdist
except ModuleNotFoundError:
    pass
else:

    @pytest.hookimpl(wrapper=True)
    def pytest_xdist_auto_num_workers(config: pytest.Config) -> Generator[None, int, int]:
        return min((yield), len(config.option.file_or_dir))

which works most of the time, but if the user runs pytest with -p no:xdist, it will crash because the xdist module exists but the plugin is disabled, causing it to attempt to register the hook when pytest would not recognize it.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the conditional pytest_xdist_auto_num_workers registration shown in the issue, both with xdist enabled and with -p no:xdist. Read pytest's optional third-party hook documentation and trace when pytest_xdist_auto_num_workers runs relative to pytest_configure. Done means the hook is invoked when xdist is enabled and does not cause an unknown-hook failure when xdist is disabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.