dask / dask/dask-gateway

Decide on `wait_for_workers` implementation in client cluster object

Open
#798 0 comments 0 reactions 0 assignees View on GitHub
maintenance
Dominant language
Python
Stars
148
Forks
93
PR merge metrics
No merged PRs in 30d

Description

Should the [dask-gateway client's GatewayCluster class](https://github.com/dask/dask-gateway/blob/ac7ecfe24e782823d4404c0bda88b887ff68038e/dask-gateway/dask_gateway/client.py#L767) implement `wait_for_workers`?

Its a question raised without knowing if its relevant or not. I understand it as it would enable the `wait_for_workers` command to get dask-gateway specific logic, such as asking the dask-gateway server directly about worker startus instead of asking the scheduler - but it could be better to just ask the scheduler directly as we do now -- because it will have the final say if a worker is ready or not anyhow right?

### Action points

- Conclude if there is a benefit to implementing such function.

### Context

- Observed issue: https://github.com/dask/dask-gateway/issues/782
- Change causing issue: https://github.com/dask/distributed/pull/6700
- Issue fix to not force having [dask-gateway client's GatewayCluster class](https://github.com/dask/dask-gateway/blob/ac7ecfe24e782823d4404c0bda88b887ff68038e/dask-gateway/dask_gateway/client.py#L767) implement `wait_for_workers`

Contributor guide

Open the contributing guide

Research direction

Start with dask_gateway/client.py at the GatewayCluster definition, then read dask-gateway#782 and distributed#6700 to understand the behavior change and current wait_for_workers path. Compare the client and scheduler responsibilities; the work is done when the benefit of a GatewayCluster implementation is decided and the rationale is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.