pytest-dev / pytest-dev/pytest-forked

Silent failures when a worker dies (e.g. segfault) and using forked and xdist

Open
#27 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
83
Forks
29
PR merge metrics
No merged PRs in 30d

Description

The following test case fails when both --forked and xdist (-n, 2) is used. It passes if only --forked is used.

import pytest


def test_segfault_reported_as_failure(testdir):
    p1 = testdir.makepyfile("""
    import ctypes

    def test_with_segfault():
        ctypes.string_at(0)

    def test_that_passes():
        assert True
    """)
    result = testdir.runpytest(p1, '--forked', '-n', 2)  # this doesn't fail if we remove '-n', 2
    result.stdout.fnmatch_lines([
        "*1 failed*"
    ])

with the following output:

======================================== test session starts =========================================
platform linux -- Python 3.7.5, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /home
plugins: xdist-1.30.1.dev1+g79dd52b, forked-1.1.3
collected 1 item

home/test_segfault.py Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/_pytest/pytester.py", line 863, in runpytest_inprocess
    reprec = self.inline_run(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/_pytest/pytester.py", line 829, in inline_run
    ret = pytest.main(list(args), plugins=plugins)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 71, in main
    config = _prepareconfig(args, plugins)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 221, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 92, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 86, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/local/lib/python3.7/site-packages/_pytest/helpconfig.py", line 89, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 736, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 943, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 878, in _preparse
    self._initini(args)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 803, in _initini
    args, namespace=copy.copy(self.option)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/argparsing.py", line 128, in parse_known_and_unknown_args
    return optparser.parse_known_args(args, namespace=namespace)
  File "/usr/local/lib/python3.7/argparse.py", line 1787, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/local/lib/python3.7/argparse.py", line 1828, in _parse_known_args
    option_tuple = self._parse_optional(arg_string)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/argparsing.py", line 377, in _parse_optional
    if not arg_string[0] in self.prefix_chars:
TypeError: 'int' object is not subscriptable
F

My execution environment is in the following docker container:

FROM python:3.7-alpine

RUN apk add --no-cache --virtual .build-deps git

RUN pip3 install git+https://github.com/pytest-dev/pytest-forked

RUN pip3 install git+https://github.com/pytest-dev/pytest-xdist

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce test_segfault_reported_as_failure with the shown testdir.runpytest call using --forked and -n 2, then compare it with the --forked-only run. Start by tracing pytester.runpytest_inprocess and the worker-failure handling involved in pytest-forked and pytest-xdist. Done means a worker segfault is reported as one failed test instead of a silent failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.