dask / dask/dask-jobqueue

Document common cluster specific (but not job scheduler specific) quirks with work-arounds if available

Open
#356 3 comments 0 reactions 0 assignees View on GitHub
documentation
Dominant language
Python
Stars
256
Forks
150
PR merge metrics
No merged PRs in 30d

Description

I wish there were some list of cluster configuration quirks (that are not job scheduler specific) and possible work-arounds (when there are some) somewhere in the doc (I was not aware of limitation of TCP/IP connection limitations between login and compute nodes in some clusters until a few days ago). Here are a rough list off the top of my head:

* submit_command not available on the compute nodes, e.g. #333. Possible work-around: https://github.com/dask/dask-jobqueue/issues/333#issuecomment-530263090 (I never tried it myself). This is the case for all the OAR clusters I know about, i.e. the submit command is never available on the compute nodes so in principle I could test this idea.
* TCP/IP restrictions between login and compute nodes e.g. #354 and #355. Possible work-around: start the main script / notebook in an interactive node with all the additional pain and limitations this entails, see https://github.com/dask/dask-jobqueue/issues/354#issuecomment-542879534 for the one I know about.
* non uniform network interfaces on login and compute nodes. I guess same work-around as TCP/IP restriction would work but not a great work-around.

Please add more if you know more off the top of your head.

cc @mrocklin @guillaumeeb @jhamman

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.