dask / dask/distributed

Narrow down `except OSError` blocks

Open
#4,989 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

In case of timeout, the distributed code currently raises a generic OSError (search for: ``raise OSError``).
There are many lines that contain ``except OSError`` or variations thereof (search for regex: ``except .*OSError``.

In some cases, we have checks (in production, not in the unit tests) on the string repr of the caught OSError to figure out if they are actually timeouts.

As a general design rule, it pays to be as specific as possible when raising and catching exceptions.

I don't recommend changing the timeouts to TimeoutError, because:
a. TimeoutError is not a subclass of OSError, so many current except blocks would stop working and one would need to chase them *all* down
b. it creates confusion with asyncio.TimeoutError.

I would suggest instead creating a new exception ``class ConnectionTimeoutError(OSError): pass``

We should search for all ``except .*OSError`` and investigate, case by case, if it's possible to narrow it down to some of its subclasses (``except (CommClosedError, ConnectionTimeoutError, RPCClosed)`` hopefully?)
This of course carries the risk of getting overzealous and accidentally resuscitating some old infrequent crash caused by *other* subclasses of OSError (unhandled failures from the socket module?).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.