dask / dask/dask-jobqueue

SGECluster in multiple queues

Open
#378 3 comments 0 reactions 0 assignees View on GitHub
enhancement question
Dominant language
Python
Stars
256
Forks
150
PR merge metrics
No merged PRs in 30d

Description

Hi,

I'm opening this issue here, because I'm not having much success with stack overflow (https://stackoverflow.com/questions/59391000/sgecluster-in-multiple-queues). Sorry for the noise.

I'm using `dask.distributed` to launch jobs on a SGE cluster (https://jobqueue.dask.org/en/latest/generated/dask_jobqueue.SGECluster.html#dask_jobqueue.SGECluster) via `dask.bags` and/or `dask.delayed`.

Everything works nicely. However, I may have some `dask.delayed` that should run in a specific queue on my SGE cluster (due to GPU availability for instance (or some other requirement)). How can I work this out on dask?

In other words, how can I define a dask_jobqueue.SGECluster with multiple queues and/or different resource specs? dask_jobqueue.SGECluster allows me to configure one cluster setup only (one queue, one resource spec, etc..).

Thanks

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.