pytest-dev / pytest-dev/pytest-xdist

Required options break child processes

Open
#383 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

$ cat test_stuff.py
def test_stuff():
    pass

$ cat conftest.py
import pytest
@pytest.hookimpl
def pytest_addoption(parser):
    parser.addoption("--my-option", required=True)

$ py.test --my-option=abcd
============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-4.0.1, py-1.7.0, pluggy-0.8.0
rootdir: /tmp/test, inifile:
plugins: xdist-1.24.1, forked-0.2
collected 1 item                                                               

test_stuff.py .                                                          [100%]

=========================== 1 passed in 0.01 seconds ===========================

$ py.test --my-option=abcd -n2
============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-4.0.1, py-1.7.0, pluggy-0.8.0
rootdir: /tmp/test, inifile:
plugins: xdist-1.24.1, forked-0.2
gw0 C / gw1 Cusage: -c [options] [file_or_dir] [file_or_dir] [...]
-c: error: the following arguments are required: --my-option
[gw0] node down: Traceback (most recent call last):
  File "/tmp/venv/lib64/python3.6/site-packages/execnet/gateway_base.py", line 1072, in executetask
    do_exec(co, loc)  # noqa
  File "<string>", line 1, in do_exec
  File "<remote exec>", line 263, in <module>
  File "<remote exec>", line 239, in remote_initconfig
  File "/tmp/venv/lib64/python3.6/site-packages/_pytest/config/__init__.py", line 704, in fromdictargs
    config.parse(args, addopts=False)
  File "/tmp/venv/lib64/python3.6/site-packages/_pytest/config/__init__.py", line 851, in parse
    args, self.option, namespace=self.option
  File "/tmp/venv/lib64/python3.6/site-packages/_pytest/config/argparsing.py", line 100, in parse_setoption
    parsedoption = self.parse(args, namespace=namespace)
  File "/tmp/venv/lib64/python3.6/site-packages/_pytest/config/argparsing.py", line 80, in parse
    return self.optparser.parse_args(args, namespace=namespace)
  File "/tmp/venv/lib64/python3.6/site-packages/_pytest/config/argparsing.py", line 348, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib64/python3.6/argparse.py", line 1762, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib64/python3.6/argparse.py", line 1997, in _parse_known_args
    ', '.join(required_actions))
  File "/tmp/venv/lib64/python3.6/site-packages/_pytest/config/argparsing.py", line 344, in error
    self.exit(EXIT_USAGEERROR, _("%(prog)s: error: %(message)s\n") % args)
  File "/usr/lib64/python3.6/argparse.py", line 2376, in exit
    _sys.exit(status)
SystemExit: 4

Replacing crashed worker gw0

The crash and replacement of worker repeats ad infinitum until the process is killed with Ctrl+C.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the commands using conftest.py, test_stuff.py, and pytest_addoption with -n2. Trace worker configuration initialization around the _pytest/config/init.py entries shown in the traceback and determine how required options reach child processes. Done means required options no longer crash and endlessly replace workers, while the single-process and distributed examples still pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.