Batchtools interacts poorly with packrat (probably renv also)
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
Research direction
Start at doJobCollection and trace how batchtools sets the job's initial and analysis working directories. Reproduce the issue with a packrat project and compare the parent session's .libPaths() with those available in the job. Done means packrat initializes in the analysis directory and the parent library paths are preserved.
Written by the indexing model from the issue text.
Description
It seems like batchtools and packrat should be usable together for reproducible HPC. Packrat package environments help keep versions consistent within a project. Unfortunately batchtools doesn't work naturally with packrat.
Packrat intializes itself with a .Rprofile file, which must be visible from the analysis directory. Jobs submitted with batchtools start in the registry directory before moving to the analysis directory (or specified work dir). So packrat is never initialized and isolation is broken.
Additionally, the .libPaths() for the parent session aren't preserved in jobs, so any modification a user does to their path will potentially cause problems for the jobs.
This can be fixed either by changing directories prior to initiating the job (possibly by suggesting users add cd commands to their submit scripts), or by having doJobCollection import the parent session's lib paths.
- Dominant language
- R
- Stars
- 184
- Forks
- 53
- Avg merge
- 7d 2h
- Merged PRs (30d)
- 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from mlr-org/batchtools
-
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
mlr-org/batchtools#321 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
mlr-org/batchtools#317 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
mlr-org/batchtools#315 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 45/100
mlr-org/batchtools#305 · 2 comments · 1 reaction ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 48/100
mlr-org/batchtools#304 · 1 comment ·
All issues in mlr-org/batchtools
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
r-lib/pkgdepends#485 · 3 comments ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
beginners blocker
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enviPathR OpenBuild Error Build OK Build Warning policies-accepted pre-review precheck-passed
Difficulty 1/5 Under an hour Newbie friendliness 84/100
Bioconductor/BiocContributions#207 · 6 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
datacarpentry/semester-biology#1255 ·