Client returned by distributed.get_client() on LocalCluster worker has unknown properties
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
The client returned by distributed.get_client() on a LocalCluster worker appears to be the same client used by the original client which created the cluster, as opposed to a standard synchronous client. For example, if you create a local cluster using AioClient() or Client(asynchronous=True), it appears as if the result of distributed.get_client() within a synchronous function called in a task will be a reference to that object. This is a problem because, as you are now in a synchronous context in a running event loop, it becomes hard to write code which could be run on both a LocalCluster and a connection to a remote cluster. Is this intended behavior? As a workaround, I'm calling the following to ensure that the client is always the same, although one would expect this to have some overhead:
```python
from distributed import get_client, Client
def f():
existing = get_client() # This may be an AioClient, or a Client() or a Client(asynchronous=True)
client = Client(address=existing.scheduler.address)
# do things with client
client.close()
```
Contributor guide
Assessment
This issue has not been assessed yet.