pytest-dev / pytest-dev/pytest-xdist

Pytest-xdist doen't recognise custom plugin that is executed as pytest.main(sys.argv[1:], CustomPlugin())

Open
#671 4 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

I have the following module which acts as a script to execute pytest.

class CustomPlugin:

    def pytest_configure(self,config):

        config.addinivalue_line(
            "python_files", "*.py"
        )
        config.addinivalue_line(
            "markers", "ignore_stdout: mark test to ignore stdout as failed task"
        )
        config.addinivalue_line(
            "markers", "ignore_stderr: mark test to ignore stderr as failed task"
        )
        config.addinivalue_line(
            "markers", "big: mark test as big and only running it certain times"
        )

    def is_setup_py(self,path):
        if path.basename != "setup.py":
            return False
        contents = path.read_binary()
        return b"setuptools" in contents or b"distutils" in contents
        
    def pytest_addoption(self,parser):
        parser.addoption("--big", dest="exclude_big",default=True, action='store_false', help='run big test')
        parser.addoption("--no-big", dest="exclude_big", action='store_true', help='miss big test [ default ]' )

    def pytest_collection_modifyitems(self,config, items):
        exclude_big = config.getoption("exclude_big")
        deselect_tests, selected = [],[]
        for item in items:
            if exclude_big and "big" in item.keywords or item.name == "test_ignore_this":
                deselect_tests.append(item)
            else:
                selected.append(item)
        items[:] = selected
        config.hook.pytest_deselected(items=deselect_tests)

    @pytest.hookimpl(hookwrapper=True)
    def pytest_runtest_makereport(self, item,call):
        report = (yield).get_result()

        markers = [mark.name for mark in item.own_markers if mark.name in ['xfail','ignore_stdout','ignore_stderr']]
        
        if report.when == 'call' \
            and len(item._report_sections)>0 \
                and len(item._report_sections[0])==3 \
                    and 'ignore_stdout' not in markers \
                        and 'ignore_stderr' not in markers:
            if len(markers) > 0:
                report.outcome = "skipped"
            else:    
                report.outcome = "failed"


if __name__=='__main__':
    pytest.main(sys.argv[1:], plugins=[CustomPlugin()])

The script works as expected when I run it normally without pytest-xdist.

But when I use pytest-xdist it is not able to recognise the custom-plugin based markers and collection_modifications.

Attached the outputs while running normally as well as while using xdist

Output while executing normally

Output while executing using xdist

How to make pytest-xdist run the custom-plugin while executing the tests parallely?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the script entry point containing pytest.main(sys.argv[1:], plugins=[CustomPlugin()]) and compare the attached normal and xdist outputs. Trace how CustomPlugin is made available during worker execution, then verify that its markers and collection modifications are applied when tests run in parallel.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.