pytest-dev / pytest-dev/pytest-xdist
`gevent` execmode hangs in python 3.7.5
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 288
- Avg merge
- 9h 30m
- Merged PRs (30d)
- 2
Description
The problem only occurs at python3.
Environment:
- OS:
Linux faa410b1f09b 4.19.121-linuxkit #1 SMP Thu Jan 21 15:36:34 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux - Python version:
Python 3.7.5
Dependencies:
apipkg 1.5
attrs 20.3.0
execnet 1.8.0
gevent 21.1.2
greenlet 1.0.0
iniconfig 1.1.1
packaging 20.9
pip 19.2.3
pluggy 0.13.1
py 1.10.0
pyparsing 2.4.7
pytest 6.2.3
pytest-forked 1.3.0
pytest-xdist 2.2.1
setuptools 41.2.0
toml 0.10.2
zope.event 4.5.0
zope.interface 5.4.0
Command: pytest -d --tx popen//execmodel=gevent test.py
Code:
from gevent import monkey
monkey.patch_all()
def test_assert_0():
assert 0
def test_assert_1():
assert 1
In Python 2 environment command runs all tests, reports failure on test_assert_0 and ends. In Python 3 environment test hangs before scanning for tests.
Execnet debug logs:
====================================== test session starts =======================================
platform darwin -- Python 3.8.2, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: /Users/changwan.park/Documents/py3_venv
plugins: xdist-2.2.1, forked-1.3.0
gw0 C[13946] gw0 [receiver-thread] RECEIVERTHREAD: starting to run
[13946] gw0 sent <Message CHANNEL_EXEC channel=1 lendata=8631>
[13946] gw0 sent <Message CHANNEL_DATA channel=1 lendata=470>
[13948] creating workergateway on <execnet.gateway_base.Popen2IO object at 0x10dc89bb0>
[13948] gw0-worker [serve] spawning receiver thread
[13948] gw0-worker [serve] joining receiver thread
[13948] gw0-worker waiting for receiver thread to finish
[13948] gw0-worker [receiver-thread] RECEIVERTHREAD: starting to run
[13948] gw0-worker [receiver-thread] received <Message CHANNEL_EXEC channel=1 lendata=8631>
[13948] gw0-worker [receiver-thread] received <Message CHANNEL_DATA channel=1 lendata=470>
Different issue compared with https://github.com/pytest-dev/execnet/issues/75. By comparing execnet debug trace, above example hangs after gw0-worker only receives gw0but does not execute the received command.
After some digging, I found out the hanging occurres since remote_exec hangs. See https://github.com/pytest-dev/execnet/issues/123.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with the shown pytest command and test.py under Python 3.7.5, then compare the execnet debug trace with the Python 2 behavior. Start by reading the referenced execnet remote_exec issue and the gevent execmode path; done means the command no longer hangs before test collection and reports the expected failing assertion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100