dask / dask/distributed

Duplicated client instance if scheduler host is dns name and not ip

Open
#2,517 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Versions:
```
distributed: ==1.25.3
dask: ==1.1.1
```

Python: 3.7.1

The following code works well

```python
import uuid

from distributed import Client

client = Client('127.0.0.1:8786')

def create_dataset():
return {
'output1': 'xyz',
'output2': 123,
}

ref = uuid.uuid4()
client.publish_dataset(client.submit(create_dataset), name=ref.hex)
outputs = client.get_dataset(ref.hex)
print((id(client), id(outputs.client)))
output1 = client.submit(lambda o: o['output1'], outputs)
print(output1.result())

```

Output:

```
(4432509136, 4432509136)
xyz
```

But if instead of using IP as scheduler host I use a hostname (tested locally by editing /etc/hosts and adding the following line: `127.0.0.1 dask.local`):

```python
import uuid

from distributed import Client

client = Client('dask.local:8786')

def create_dataset():
return {
'output1': 'xyz',
'output2': 123,
}

ref = uuid.uuid4()
client.publish_dataset(client.submit(create_dataset), name=ref.hex)
outputs = client.get_dataset(ref.hex)
print((id(client), id(outputs.client)))
output1 = client.submit(lambda o: o['output1'], outputs)
print(output1.result())
```

It hangs for a few seconds and then outputs:

```
(4477835472, 4497450824)
Traceback (most recent call last):
File "dask_test_1.py", line 19, in
output1 = client.submit(lambda o: o['output1'], outputs)
File "/Users/mateusz/.virtualenvs/flipper-api-oRORWyHy/lib/python3.7/site-packages/distributed/client.py", line 1278, in submit
actors=actor)
File "/Users/mateusz/.virtualenvs/flipper-api-oRORWyHy/lib/python3.7/site-packages/distributed/client.py", line 2228, in _graph_to_futures
raise ValueError(msg)
ValueError: Inputs contain futures that were created by another client.
```

As you can see, original client instance and the one assigned to `outputs` are different, and submitting `output1` throws an exception.

I found a workaround for that, I can simply resolve hostname with:

```python
client = Client(f"{socket.gethostbyname('dask.local')}:8786")
```
and then it works fine, but it's annoying.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.