dask / dask/distributed

Generic Cluster object for when you just have a scheduler address

Open
#6,791 2 comments 1 reaction 0 assignees View on GitHub
discussion
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

This came up in the dev meeting today, the idea is that there could be a Cluster object that behaves like the `ProxyCluster` in dask-ctl. This cluster would accept a scheduler address and not have any mechanisms for doing cluster management (like `scale`, or `close`).

One possible name for this would be `RemoteCluster`. The benefit of such a cluster is that it would allow us to do a clearer separation between cluster and client. So for instance the best practice for connecting to a generic external cluster would be:

current
```python
from distributed import Client

client = Client("tcp://192.168.0.184:8788")
```
proposed
```python
from distributed import RemoteCluster

client = RemoteCluster("tcp://192.168.0.184:8788").get_client()
```

The hope is that this syntax will make it easier for people to use different types of clusters and understand the client/cluster division of labor.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.