[Feature] Add worker capabilities/labels
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
It would be really nice if you could add something like capabilities/labels (however you want to call them) to `dask`
It would be something like:
```
dask-worker scheduler:8786 --capabilities "vaapi,nodejs"
dask-worker scheduler:8786
```
Where you could schedule a task like:
```python
add_t = client.submit(add, 1, 4)
video_t = client.submit(process, "/path/to/file", capabilities=["vaapi"])
```
So that certain tasks only get scheduled on certain workers, which support that kind of task
Additionally another feature to consider would be enum-like capabilities:
```
dask-worker scheduler:8786 --capabilities "some_feature,arch=amd64"
dask-worker scheduler:8786 --capabilities "arch=armv8"
```
With a python equivalent of:
```python
result_t = client.submit(process, some_data, capabilities={"some_feature": True, "arch": "amd64"})
other_result_t = client.submit(process, other_data, capabilities=["some_feature"])
```
The current _equivalent_ of capabilities would be something like:
```
dask-worker scheduler:8786 --resources "vaapi=9999"
```
Contributor guide
Assessment
This issue has not been assessed yet.