makeRegistry(NA) causes jobs to expire on Slurm cluster
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 184
- Forks
- 53
- Avg merge
- 7d 2h
- Merged PRs (30d)
- 1
Description
Hi,
I've just been working on setting up batchtools on a Slurm cluster. I was finding that all my jobs were listed as 'Expired' after running getStatus() and it took a while to figure out why. Seems to be because I was following the 'Getting Started' vignette, which suggests running:
makeRegistry(NA)
But that creates the registry in a temporary folder. So far as I understand, the temporary folder gets created on the login node, and is not accessible from the worker nodes. So once the worker nodes are called, they cannot access the job files, and so expire (without being able to write a log file explaining what went wrong).
I don't have access to any other Slurm clusters, so don't know if this is the default behaviour with Slurm clusters or something specific to mine. To save other people running through this, could I suggest either (A) changing the 'Getting started' guide to have makeRegistry() use a random folder in the home directory, or (B) change the behaviour of makeRegistry on Slurm clusters to avoid temporary folders?
Thanks for developing this great package!
Nathan
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Getting Started vignette and the makeRegistry(NA) and getStatus() workflow described in the issue. Reproduce the registry location behavior on a Slurm cluster, then determine whether the documented example or makeRegistry behavior is the appropriate scope; done means jobs no longer expire because workers cannot access the registry files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- hpc
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100