Preserve hostnames in worker addresses
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
The scheduler internally converts an address with a hostname, e.g.,
```
tls://worker1.example.com:8786
```
into an IP address:
```
tls://129.93.2.2:8786
```
For the `tcp://` protocol, this is fine -- probably saves a few cycles in address translation.
However, for `tls://`, this drops an important piece of information: the desired hostname. A TLS router (such as Traefik) can use the desired hostname information to internally route requests to the right location.
For example, we run a Dask cluster per user that's partially split across Kubernetes and a HTCondor cluster. The scheduler is run behind a TLS router (Traefik) which exposes the scheduler externally. A client request to connect to the scheduler `tls://scheduler1.example.com` preserves the hostname. Hence, SNI (https://en.wikipedia.org/wiki/Server_Name_Indication) allows the router to proxy the client to the correct scheduler. We do this as we don't have sufficient public IPs to allocate one to each scheduler. [Yes, we are aware of dask-gateway; it doesn't quite fit our needs - I don't want to derail the ticket on that though.]
Given the majority of the data comes from a distributed storage system and we aren't trying to push GB/s through the proxy, this setup with the scheduler works swimmingly well.
However, it doesn't work for the workers as the scheduler coerces the address of the worker almost immediately to an IP address.
SO:
1. Is there a good reason why this is done? It seems purposeful; what's the advantage that using IPs conveys?
2. Would there be interest in a patch that preserves the hostname instead? [Preserving the hostname might also convey some advantage to dual IPv6/IPv4 hosts]
3. If not, what about keeping both? That is, address based on IP but keep around the hostname so it can be passed to the TLS layer.
Contributor guide
Assessment
This issue has not been assessed yet.