dask / dask/distributed

CommClosedError thrown by python script but not by IPython notebook

Open
#934 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Hi

I get the following error if I run a `dask.distributed` script but no error when I run the same code in an IPython notebook:

```
tornado.application - ERROR - Exception in callback
Traceback (most recent call last):
File "/home/beowulf/miniconda2/envs/clarke/lib/python2.7/site-packages/tornado/ioloop.py", line 604, in _run_callback
ret = callback()
File "/home/beowulf/miniconda2/envs/clarke/lib/python2.7/site-packages/tornado/stack_context.py", line 275, in null_wrapper
return fn(*args, **kwargs)
File "/home/beowulf/miniconda2/envs/clarke/lib/python2.7/site-packages/distributed/batched.py", line 102, in send
raise CommClosedError
```

This is the code:

```
from distributed import Client

client = Client("127.0.0.1:8786")

def get_hostname():
import socket
return socket.gethostname()

def dummy(x):
return (get_hostname(), )

jobs = client.map(dummy, range(10000))
total = client.submit(len, jobs)
total.result()
```

I am guessing this is happening because the script terminates before the jobs do?

Any help would be gratefully received. Thanks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.