pytest-dev / pytest-dev/execnet
`remote_exec` hangs forever when remotely using `gevent` model only at python3
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 102
- 派生
- 47
- PR 合并指标
- 30 天内没有已合并 PR
描述
The problem only occurs at python3.
Environment:
- OS:
Linux faa410b1f09b 4.19.121-linuxkit #1 SMP Thu Jan 21 15:36:34 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux - Python version:
Python 3.7.5
Dependencies:
- gevent version:
21.1.2 - execnet version:
1.8.0
Code:
import execnet
# hangs forever when remote is gevent
execnet.set_execmodel('thread', 'gevent')
gw = execnet.makegateway()
print(gw)
print(gw.remote_status())
print('channel.send')
print (gw.remote_exec("channel.send(333)").receive())
print('everything done')
Problem: Above code hangs forever.
Stdout:
<Gateway id='gw0' receive-live, thread model, 0 active channels>
<RInfo 'execmodel=gevent, numchannels=0, numexecuting=0'>
channel.send
Debug log:
[89653] gw0 sent <Message STATUS channel=1 lendata=0>
[89653] gw0 [receiver-thread] RECEIVERTHREAD: starting to run
[89655] creating workergateway on <execnet.gateway_base.Popen2IO object at 0x1106a5820>
[89655] gw0-worker [serve] spawning receiver thread
[89655] gw0-worker [serve] joining receiver thread
[89655] gw0-worker waiting for receiver thread to finish
[89655] gw0-worker [receiver-thread] RECEIVERTHREAD: starting to run
[89655] gw0-worker [receiver-thread] received <Message STATUS channel=1 lendata=0>
[89655] gw0-worker sent <Message CHANNEL_DATA channel=1 lendata=73>
[89653] gw0 [receiver-thread] received <Message CHANNEL_DATA channel=1 lendata=73>
[89653] gw0 [receiver-thread] received <Message CHANNEL_CLOSE channel=1 lendata=0>
[89655] gw0-worker sent <Message CHANNEL_CLOSE channel=1 lendata=0>
[89653] gw0 1 channel.__del__
[89653] gw0 sent <Message CHANNEL_EXEC channel=3 lendata=31>
[89655] gw0-worker [receiver-thread] received <Message CHANNEL_EXEC channel=3 lendata=31>
The code hangs while executing remote_exec. The problem only occurs when remote thread model is gevent.
I guess the problem occurred because while gevent is rapidly syncing with python3, execnet was not maintained :C.
Maybe related to function mapping when making ExecModel:
https://github.com/pytest-dev/execnet/blob/894533a515baf526c35ea0fe525b4866c6824ab8/execnet/gateway_base.py#L112-L125
I know this repo is unmaintained. Hoping for receiving someone's help!
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
使用 execnet 的 thread-to-gevent 配置以及 issue 中列出的版本,通过 Python 3 示例复现挂起问题。首先检查 execnet/gateway_base.py 中 112-125 行附近的 ExecModel 映射,然后跟踪 remote_exec 经过 receiver 和 worker 线程的过程。当远程调用返回 333,且程序在不挂起的情况下到达“everything done”时,即视为完成。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- distributed-systems
- Issue 类型
- 缺陷
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100