Distributed compute doesn't appear to respect resource requirements
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
I have attempting to ensure that only 1 particular key requires the usage of GPU, while the rest should be scheduled fairly over all other systems.
```py
import dask
import dask.distributed
def inc(x):
worker = dask.distributed.get_worker()
print(f'inc {x}')
print(worker.name)
print(worker.total_resources)
print(worker.available_resources)
return x + 1
def add(x, y):
worker = dask.distributed.get_worker()
print(f'add {x} {y}')
print(worker.name)
print(worker.total_resources)
print(worker.available_resources)
return x + y
def double(x):
worker = dask.distributed.get_worker()
print(f'double {x}')
print(worker.name)
print(worker.total_resources)
print(worker.available_resources)
return x * 2
def main():
a = dask.delayed(inc)(1)
b = dask.delayed(inc)(2)
c = dask.delayed(add)(a, b)
d = dask.delayed(double)(c)
with dask.distributed.Client('127.0.0.1:3201', set_as_default=False) as scheduler:
scheduler_info = scheduler.scheduler_info()
print('Scheduler: %s', scheduler_info['address'])
print('Status Port: %s', scheduler_info['services']['dashboard'])
future = scheduler.compute({
'a': a,
'b': b,
'c': c,
'd': d
}, resources={
'c': {
'CPU': 50
},
'd': {
'GPU': 1
}
})
result = future.result()
print('RESULT')
print(result)
input()
if __name__ == '__main__':
main()
```
When I run the above, I see that the `a`, `c` and `d` gets executed on the worker with `GPU=1`, but `b` gets executed on the worker with `CPU=100`. Note that I used `CPU=...` as a just a dummy resource to see what would happen. Most likely I will have workers that have no resource specifications, and only 1 worker process with `GPU=1`.
It seems like the resource constraints is not being obeyed explicitly. Is this how it is meant to be used.
In the above code, I would have expected that `d` gets computed on the GPU worker, but `a`, `b`, and `c` be executed on the CPU workers.
Contributor guide
Assessment
This issue has not been assessed yet.