Outstanding task on client but hub says completed when using broadcast view
- 主要言語
- Jupyter Notebook
- スター
- 2.6k
- フォーク
- 1k
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
I want to `purge_everything ` or `purge_local_results ` but I'm getting `RuntimeError: Can't purge outstanding tasks: ...`
The reason I want to purge is that after a couple of tasks my engines memory seems to get full and slows down the process. I'm trying to figure out if it's a memory issue related to ipyparallel or something related to the tasks.
Here is a rough sketch of what I'm doing:
do some setup:
```
from ipyparallel import Client, Reference
rc = Client(client_args, client_kwargs)
view = rc.broadcast_view(view_args, view_kwargs)
view.is_coalescing = False
view.apply_sync( # do some setup stuff )
```
then:
```
ar = view.apply( # do the real work, and pickle it)
fvs = np.array(sum([pickle.loads(fv) for fv in ar.get()],[]))
view.client.purge_local_results('all') # this says there are outstanding tasks
view.client.queue_status() # this says all tasks are complete
```
コントリビューションガイド
調査の方向性
Start with the broadcast_view and apply workflow shown in the report, then compare ar.get() completion with queue_status() and the purge_local_results('all') check. Done means the reported completed-task state and the purge operation agree, or the issue clearly documents the remaining limitation. No file or test is named.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- jupyter, numpy, python
- 領域
- distributed-systems
- issue の種類
- バグ
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100