Determine behavior for failed process task
Open
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
In https://github.com/dask/distributed/pull/5063 we improved support to run tasks in separate processes. However it's unclear what we should do when the task causes the process that ran the task to fail entirely. In a threaded situation this would cause the worker to fail, and for the task to be marked as suspicious (we can't be certain which task caused the error). In the process case we do know the errant task, and we currently return an exception.
Should we?
We might also consider rescheduling this task and using the standard suspicious tasks behavior. It's not immediately clear which behavior is ideal.
Contributor guide
Assessment
This issue has not been assessed yet.