pytest-dev / pytest-dev/execnet

`remote_exec` hangs forever when remotely using `gevent` model only at python3

Open
#123 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug help wanted
Dominant language
Python
Stars
102
Forks
47
PR merge metrics
No merged PRs in 30d

Description

The problem only occurs at python3.

Environment:

  • OS: Linux faa410b1f09b 4.19.121-linuxkit #1 SMP Thu Jan 21 15:36:34 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
  • Python version: Python 3.7.5

Dependencies:

  • gevent version: 21.1.2
  • execnet version: 1.8.0

Code:

import execnet
# hangs forever when remote is gevent
execnet.set_execmodel('thread', 'gevent')
gw = execnet.makegateway()
print(gw)
print(gw.remote_status())

print('channel.send')
print (gw.remote_exec("channel.send(333)").receive())
print('everything done')

Problem: Above code hangs forever.

Stdout:

<Gateway id='gw0' receive-live, thread model, 0 active channels>
<RInfo 'execmodel=gevent, numchannels=0, numexecuting=0'>
channel.send

Debug log:

[89653] gw0 sent <Message STATUS channel=1 lendata=0>
[89653] gw0 [receiver-thread] RECEIVERTHREAD: starting to run
[89655] creating workergateway on <execnet.gateway_base.Popen2IO object at 0x1106a5820>
[89655] gw0-worker [serve] spawning receiver thread
[89655] gw0-worker [serve] joining receiver thread
[89655] gw0-worker waiting for receiver thread to finish
[89655] gw0-worker [receiver-thread] RECEIVERTHREAD: starting to run
[89655] gw0-worker [receiver-thread] received <Message STATUS channel=1 lendata=0>
[89655] gw0-worker sent <Message CHANNEL_DATA channel=1 lendata=73>
[89653] gw0 [receiver-thread] received <Message CHANNEL_DATA channel=1 lendata=73>
[89653] gw0 [receiver-thread] received <Message CHANNEL_CLOSE channel=1 lendata=0>
[89655] gw0-worker sent <Message CHANNEL_CLOSE channel=1 lendata=0>
[89653] gw0 1 channel.__del__
[89653] gw0 sent <Message CHANNEL_EXEC channel=3 lendata=31>
[89655] gw0-worker [receiver-thread] received <Message CHANNEL_EXEC channel=3 lendata=31>

The code hangs while executing remote_exec. The problem only occurs when remote thread model is gevent.

I guess the problem occurred because while gevent is rapidly syncing with python3, execnet was not maintained :C.

Maybe related to function mapping when making ExecModel:
https://github.com/pytest-dev/execnet/blob/894533a515baf526c35ea0fe525b4866c6824ab8/execnet/gateway_base.py#L112-L125

I know this repo is unmaintained. Hoping for receiving someone's help!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the hang with the Python 3 example using execnet's thread-to-gevent configuration and the versions listed in the issue. Start by inspecting the ExecModel mapping in execnet/gateway_base.py around lines 112-125, then trace remote_exec through the receiver and worker threads. Done means the remote call returns 333 and the program reaches “everything done” without hanging.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.