mlr-org / mlr-org/batchtools

Slurm resource named `clusters` in docs, and `cluster` in `getClusters`

Open
#297 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
184
Forks
53
Avg merge
7d 2h
Merged PRs (30d)
1

Description

I can't quite decide which way it is.

In clusterFunctionsSlurm.R, the getClusters function is defined which looks for a resource named cluster (singular).
https://github.com/mllg/batchtools/blob/1196047ed5115d54bde2923848c1f3ec11fda6d2/R/clusterFunctionsSlurm.R#L38-L42

The documentation of submitJobs refers to clusters (plural):
https://github.com/mllg/batchtools/blob/1196047ed5115d54bde2923848c1f3ec11fda6d2/R/submitJobs.R#L25

The problem:

if this resource / getClusters is NULL, there's no cluster specified and functions like findRunning() etc. will return an empty table despite jobs are running.
For months I was wondering why my queued Slurm jobs where listed as expired and I assumed I was doing something wrong or my template was outdated.
Took me a moment to figure out that I was accidentally correctly misspecifying clusters in my resources = list(...) call 🥴

EDIT: Oh, and the template I'm using relies on resources$clusters.

Suggested fix

I'm not sure. I was about to prepare a PR when I realized that I don't want to

  • change the internal getClusters function to look for clusters instead of cluster because that will silently break peoples existing batchtools configs and they might not notice.
  • change the docs for submitJobs to (wrongly?) suggest that the resource should be named cluster, implying that multiple clusters can not be specified (which I never tried)

At the very least I thought about introducing an assertion on provided resources to shield against this sort of thing, but I'm not sure how to go about that yet.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Compare the getClusters implementation in clusterFunctionsSlurm.R with the submitJobs documentation in submitJobs.R, including how findRunning() uses the resource name and how the template uses resources$clusters. Decide how singular and plural names should remain compatible, then verify that configured running jobs are detected and the documentation and template guidance agree.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
hpc
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.