pytest-dev / pytest-dev/pytest-xdist

Version 2.0.0 may trigger internal error on Python 3

Open
#580 8 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

Hello there, we have just encountered this issue running CIs for pip on Python 3:

============================= test session starts ==============================
platform linux -- Python 3.5.3[pypy-7.0.0-final], pytest-4.6.11, py-1.9.0, pluggy-0.13.1
cachedir: .tox/pypy/.pytest_cache
rootdir: /home/travis/build/pypa/pip, inifile: setup.cfg
plugins: xdist-2.0.0, cov-2.10.1, rerunfailures-8.0, timeout-1.4.2, forked-1.3.0
timeout: 300.0s
timeout method: signal
timeout func_only: False
gw0 [2136]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/_pytest/main.py", line 206, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/_pytest/main.py", line 250, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/xdist/dsession.py", line 112, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/xdist/dsession.py", line 135, in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/xdist/dsession.py", line 238, in worker_collectionfinish
INTERNALERROR>     self.sched.schedule()
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/xdist/scheduler/load.py", line 249, in schedule
INTERNALERROR>     self._send_tests(next(nodes), 1)
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/xdist/scheduler/load.py", line 261, in _send_tests
INTERNALERROR>     node.send_runtest_some(tests_per_node)
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/xdist/workermanage.py", line 283, in send_runtest_some
INTERNALERROR>     self.sendcommand("runtests", indices=indices)
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/xdist/workermanage.py", line 299, in sendcommand
INTERNALERROR>     self.channel.send((name, kwargs))
INTERNALERROR>   File "/home/travis/build/pypa/pip/.tox/pypy/site-packages/execnet/gateway_base.py", line 728, in send
INTERNALERROR>     raise IOError("cannot send to {!r}".format(self))
INTERNALERROR> OSError: cannot send to <Channel id=1 closed>
======================== no tests ran in 11.77 seconds =========================
ERROR: InvocationError for command /home/travis/build/pypa/pip/.tox/pypy/bin/pytest --timeout 300 --use-venv -m unit -n auto (exited with code 3)

Regarding the error, I found this StackOverflow thread but I'm not sure if it is still applicable to current pytest-xdist.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the shown pytest command on Python 3, then trace the worker and scheduling path through xdist/dsession.py, xdist/scheduler/load.py, xdist/workermanage.py, and execnet/gateway_base.py. Compare the behavior with the linked Stack Overflow guidance; done means the run no longer raises the internal OSError and the tests execute normally.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.