Duplicated client instance if scheduler host is dns name and not ip
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
Versions:
```
distributed: ==1.25.3
dask: ==1.1.1
```
Python: 3.7.1
The following code works well
```python
import uuid
from distributed import Client
client = Client('127.0.0.1:8786')
def create_dataset():
return {
'output1': 'xyz',
'output2': 123,
}
ref = uuid.uuid4()
client.publish_dataset(client.submit(create_dataset), name=ref.hex)
outputs = client.get_dataset(ref.hex)
print((id(client), id(outputs.client)))
output1 = client.submit(lambda o: o['output1'], outputs)
print(output1.result())
```
Output:
```
(4432509136, 4432509136)
xyz
```
But if instead of using IP as scheduler host I use a hostname (tested locally by editing /etc/hosts and adding the following line: `127.0.0.1 dask.local`):
```python
import uuid
from distributed import Client
client = Client('dask.local:8786')
def create_dataset():
return {
'output1': 'xyz',
'output2': 123,
}
ref = uuid.uuid4()
client.publish_dataset(client.submit(create_dataset), name=ref.hex)
outputs = client.get_dataset(ref.hex)
print((id(client), id(outputs.client)))
output1 = client.submit(lambda o: o['output1'], outputs)
print(output1.result())
```
It hangs for a few seconds and then outputs:
```
(4477835472, 4497450824)
Traceback (most recent call last):
File "dask_test_1.py", line 19, in
output1 = client.submit(lambda o: o['output1'], outputs)
File "/Users/mateusz/.virtualenvs/flipper-api-oRORWyHy/lib/python3.7/site-packages/distributed/client.py", line 1278, in submit
actors=actor)
File "/Users/mateusz/.virtualenvs/flipper-api-oRORWyHy/lib/python3.7/site-packages/distributed/client.py", line 2228, in _graph_to_futures
raise ValueError(msg)
ValueError: Inputs contain futures that were created by another client.
```
As you can see, original client instance and the one assigned to `outputs` are different, and submitting `output1` throws an exception.
I found a workaround for that, I can simply resolve hostname with:
```python
client = Client(f"{socket.gethostbyname('dask.local')}:8786")
```
and then it works fine, but it's annoying.
Contributor guide
Assessment
This issue has not been assessed yet.