usage of `dask.distributed.Queue` raises ValueError: unknown address scheme 'gateway'
- Dominant language
- Python
- Stars
- 148
- Forks
- 93
- PR merge metrics
- No merged PRs in 30d
Description
**What happened**:
When i try to use `dask.distributed.Queue` on workers of a `dask-gateway` cluster i get `ValueError: unknown address scheme 'gateway'`
**What you expected to happen**:
I expect `dask.distributed.Queue` to work, as it does e.g. in a `dask.distributed.LocalCluster`
**Minimal Complete Verifiable Example**:
```sh
python -m venv ~/venv/dask-gateway
source ~/venv/dask-gateway/bin/activate
pip install dask-gateway 'dask-gateway-server[local]'
dask-gateway-server
```
```python
from dask_gateway import Gateway
from dask.distributed import Queue
gateway = Gateway("http://127.0.0.1:8000")
cluster = gateway.new_cluster()
cluster.scale(1)
client = cluster.get_client()
q = Queue("log")
def log_something():
q.put(42)
client.submit(log_something).result()
```
results in
```
Traceback (most recent call last):
File "", line 1, in
File "/home/nikolai/venv/dask-gateway/lib/python3.9/site-packages/distributed/client.py", line 227, in result
raise exc.with_traceback(tb)
File "/home/nikolai/venv/dask-gateway/lib/python3.9/site-packages/distributed/protocol/pickle.py", line 75, in loads
return pickle.loads(x)
File "/home/nikolai/venv/dask-gateway/lib/python3.9/site-packages/distributed/queues.py", line 289, in __setstate__
client = get_client(address)
File "/home/nikolai/venv/dask-gateway/lib/python3.9/site-packages/distributed/worker.py", line 3615, in get_client
address = comm.resolve_address(address)
File "/home/nikolai/venv/dask-gateway/lib/python3.9/site-packages/distributed/comm/addressing.py", line 171, in resolve_address
backend = registry.get_backend(scheme)
File "/home/nikolai/venv/dask-gateway/lib/python3.9/site-packages/distributed/comm/registry.py", line 81, in get_backend
raise ValueError(
ValueError: unknown address scheme 'gateway' (known schemes: ['inproc', 'tcp', 'tls', 'ucx', 'ws', 'wss'])
```
**Anything else we need to know?**:
Noticed the issue in a kubernetes-deployed setup on google cloud, but it is reproducible with the local installation as well.
The code works with a distributed cluster, e.g.
```python
from dask.distributed import Client, Queue
client = Client()
q = Queue("log")
def log_something():
q.put(42)
client.submit(log_something).result()
q.get()
```
**Environment**:
- Dask version: `2021.6.2`,
- dask-gateway version: `0.9.0`
- Python version: `3.9.5`
- Operating System: arch linux
- Install method (conda, pip, source): pip
Contributor guide
Assessment
This issue has not been assessed yet.