dask / dask/dask-jobqueue

Limit number of jobs in queue or running with dask-jobqueue

Open
#365 8 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
256
Forks
150
PR merge metrics
No merged PRs in 30d

Description

Original problem raised by @jmichel-otb in https://github.com/dask/dask-jobqueue/issues/196#issuecomment-548412595.

> I currently encounter the limitation of maximum number of jobs in queue. Our maximum number of jobs in queue is 100, which is not much.
>
> I understand that using job arrays is off the table, but maybe we could make dask jobqueue aware of those limits, and inform it that it needs to start say 200 jobs but it needs to ensure that there are never more than 100 queuing ?
>
> I am not familiar with jobqueue code but I expect this to be a much smaller patch than enabling job arrays.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.