Decide on `wait_for_workers` implementation in client cluster object
- Dominant language
- Python
- Stars
- 148
- Forks
- 93
- PR merge metrics
- No merged PRs in 30d
Description
Should the [dask-gateway client's GatewayCluster class](https://github.com/dask/dask-gateway/blob/ac7ecfe24e782823d4404c0bda88b887ff68038e/dask-gateway/dask_gateway/client.py#L767) implement `wait_for_workers`?
Its a question raised without knowing if its relevant or not. I understand it as it would enable the `wait_for_workers` command to get dask-gateway specific logic, such as asking the dask-gateway server directly about worker startus instead of asking the scheduler - but it could be better to just ask the scheduler directly as we do now -- because it will have the final say if a worker is ready or not anyhow right?
### Action points
- Conclude if there is a benefit to implementing such function.
### Context
- Observed issue: https://github.com/dask/dask-gateway/issues/782
- Change causing issue: https://github.com/dask/distributed/pull/6700
- Issue fix to not force having [dask-gateway client's GatewayCluster class](https://github.com/dask/dask-gateway/blob/ac7ecfe24e782823d4404c0bda88b887ff68038e/dask-gateway/dask_gateway/client.py#L767) implement `wait_for_workers`
Contributor guide
Research direction
Start with dask_gateway/client.py at the GatewayCluster definition, then read dask-gateway#782 and distributed#6700 to understand the behavior change and current wait_for_workers path. Compare the client and scheduler responsibilities; the work is done when the benefit of a GatewayCluster implementation is decided and the rationale is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100