dask / dask/distributed

Determine behavior for failed process task

Open
#5,075 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

In https://github.com/dask/distributed/pull/5063 we improved support to run tasks in separate processes. However it's unclear what we should do when the task causes the process that ran the task to fail entirely. In a threaded situation this would cause the worker to fail, and for the task to be marked as suspicious (we can't be certain which task caused the error). In the process case we do know the errant task, and we currently return an exception.

Should we?

We might also consider rescheduling this task and using the standard suspicious tasks behavior. It's not immediately clear which behavior is ideal.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.