dask / dask/distributed

Semaphore and other objects as task condition

Open
#4,104 25 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

When you are using the builtin Semaphore to limit task concurrency of a certain type, there are currently two behaviours you can choose:
- request the semaphore: this will hold a worker while the semaphore is not acquirable, which means workers are doing nothing (running a task which is waiting for a semaphore) because they are all waiting for the same semaphore
- request the semaphore after seceding from the worker, then rejoin: this will still have a similar issue as above, although less severe/less likely (I think). This leads to the possibility of having a task scheduled, then secede for the semaphore, start a different task on the same worker, acquire the semaphore but waiting to rejoin the worker. In the mean time a different worker starts a different task which tries to acquire the semaphore but can't because of the blocked task holding the semaphore. I have seen this happening (although it might only be possible if you have 2 different semaphore's, and due to a configuration error there were less workers available then intended)

Unless there is a different option, the only solution would be to make it possible to add a semaphore or similar primitives to be added as a condition to a task (like the result of another task is a condition before the task can be scheduled).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.