dask / dask/distributed

Make remote tracebacks easier to read and distinguish

Open
#4,880 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

When an exception occurs on the cluster, the traceback that shows up is long, and includes both a local component and a remote component. I suspect this is a stumbling block for beginners because:
1. When tracebacks are too long, people glaze over and don't read them
2. For those that do read them, separating the local client boilerplate from the remote part requires practice

Specifically, I always skim for something like `raise exception.with_traceback(traceback)` in `distributed/client.py`, ignore everything above it, and just look at the remote part of the traceback. Could we format our error messages differently so users don't have to learn this unintuitive skill?

Goals:
1. Users can easily tell whether an exception happened locally or on the cluster
2. Minimal internal distributed code is shown in tracebacks when we know the error wasn't an internal distributed error.

We already do a good job with 2 on the worker side thanks to [`get_traceback`](https://github.com/dask/distributed/blob/4f83686b9ffed4e8c41bbf002c1bab24b38b9dab/distributed/utils.py#L732-L744), which removes irrelevant frames. So this might be as simple as `raise ... from None` on the client when re-raising a remote exception. Plus somehow adding a prefix like:

```
------------------------------------------------------------------------------------------------
ValueError (remote) Traceback from cluster (most recent call last)

While calling Client.compute, this error occurred on worker 'worker-abcde'
while executing task ('map-blocks-12345', 0, 0):
```
so that remote exceptions are easily distinguishable from local ones.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.