pytest-dev / pytest-dev/pytest-xdist

exit code 0 returned to pytester when an internal error occurs inside a `pytest_runtest_protocol` hook

Open
#1,017 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

i'm not sure if this is an issue with pytest-xdist or pytester, but when testing a pytest plugin that raises an exception with pytest-xdist, pytester incorrectly says the exit code was 0, when it should be 3.

from pytest import ExitCode, Pytester

def test_asdf(pytester: Pytester):
    pytester.makeconftest(
        """
        def pytest_runtest_protocol():
            raise Exception("asdf")
        """
    )
    pytester.makepyfile(
        """
        def test_foo(): ...    
        """
    )
    result = pytester.runpytest("-n", "2")
    assert [
        line for line in result.outlines if line.startswith("INTERNALERROR>")
    ]  # passes
    assert result.ret != ExitCode.OK  # fails

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the shown Pytester example, focusing on pytest_runtest_protocol, pytester.runpytest, and the -n 2 xdist path. Trace how the internal error is reported and how result.ret is populated. Done means the existing internal-error assertion still passes and result.ret reports the expected nonzero exit code, specifically 3.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.