ipython / ipython/ipyparallel

kill engines by pid & client.shutdown blocks

未關閉
#229 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
bug
主要語言
Jupyter Notebook
星號
2.6k
分支
1k
PR 合併指標
30 天內沒有已合併 PR

描述

Hey @minrk and folks, in Issue #141 you recommend to use os.kill to interrupt engines by pid. I've been trying this approach, however, it seems that when engines are killed by pid you can no longer stop the ipcluster instance using `client.shutdown` without it blocking indefinitely. Is there a better way? I'd like to be able to interrupt all running jobs if, for example, a KeyboardInterrupt is raise, and then still be able to close the ipcluster instance remotely.

Here is an example where I run this first block of code followed by either of the following two code blocks:
```
import ipyparallel as ipp
import signal
import time
import os

## open client view
client = ipp.Client()

## get engines pids
engine_pids = client[:].apply(os.getpid).get_dict()
```

If I interrupt the engines then `shutdown` blocks indefinitely:
```
## submit a job to run for a while
for i in range(4):
async = client[i].apply(time.sleep, 30)

## interrupt each engine job before it finishes
for eng in engine_pids:
os.kill(engine_pids[eng], signal.SIGINT)

## kill the ipcluster instance
client.shutdown(hub=True, block=False)
```

Or if I just try `shutdown` without interrupting engines it doesn't block indefinitely, but it does still block for the full 30 seconds, whether or not I tell it `block=False`.
```
## submit a job to run for a while
for i in range(4):
async = client[i].apply(time.sleep, 30)

## soft shutdown
client.shutdown(hub=True, block=False)
```

貢獻指南

開啟貢獻指南

研究方向

Reproduce the issue using the shown ipyparallel Client example, comparing client.shutdown(hub=True, block=False) with and without os.kill and SIGINT. Start by tracing the shutdown behavior after interrupted engine jobs; done means remote shutdown returns without hanging indefinitely or waiting for the full job duration.

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
distributed-systems
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
停滯
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。