dask / dask/distributed

Address resolution not consistent

Open
#5,915 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Dask typically can work with fully resolved host addresses (`127.0.0.1:1234` ) or hostnames (`localhost:1234`) interchangeably by resolving the hostname where necessary.

We have various toggle over the code base controlling this behaviour
- [`Scheduler.coerce_address`](https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/scheduler.py#L7539)
- [`Scheduler.heartbeat_worker`](https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/scheduler.py#L4264-L4275)
- [`Scheduler.add_worker`](https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/scheduler.py#L4370-L4393)
- [`distributed.worker.get_client`](https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/worker.py#L4383)

They are all accessing `distributed.com.addressing.resolve_address` which then dispatches the resolve to the used network backend based on the URL scheme.

We're not very consistent in this behaviour and I believe that if we want to resolve addresses, we should resolve *all* addresses as soon as the user passes them in.

Particularly, where we're not resolving addresses are
- [`Client.__init__`](https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/worker.py#L4383) and [here](https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/client.py#L891)
- `Nanny.__init__`
- `Worker.__init__`

This can cause confusing and wrong behaviour whenever we're comparing addresses, e.g.
- https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/actor.py#L173-L177
- [`Client._gather`](https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/client.py#L1991-L1992)
- [`Client._scatter`](https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/client.py#L2222-L2223)
- [serialized `Queues`](https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/queues.py#L278-L281)
- [serialized `Variables`](https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/queues.py#L242)
- [`Worker._get_client`](https://github.com/dask/distributed/blob/e1e4385893301dceabee9582b60db0190368f4e8/distributed/worker.py#L4296-L4306) (This causes the failure in https://github.com/dask/distributed/pull/5901#issuecomment-1061944494)
- ...

Side note: There is an ambiguous, not documented `Scheduler.coerce_hostname` which is not connected to address resolution but rather checks the internal list of aliases

- I would suggest to remove the kwargs in most of the functions and replace it with a config toggle, if we want to allow toggling.
- We should ensure that this behaviour is consistent, e.g. all addresses are always resolved or never

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.