futureverse / futureverse/future.batchtools
Problem forwarding batchtools resources to individual futures
- Dominant language
- R
- Stars
- 87
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
When I supply a `resources` list to a `batchtools` future, the resources seem to be ignored. I was alerted to this via `targets`, https://github.com/ropensci/targets/discussions/562#discussioncomment-1304379 and https://github.com/ropensci/targets/discussions/632 (cc @wresch, @sdechaumet)
## Reproducible example
Using an SGE cluster with this template:
```
#!/bin/bash
#$ -cwd
#$ -j y
#$ -o <%= log.file %>
#$ -V
#$ -N <%= job.name %>
#$ -pe smp <%= resources$slots %>
module load R/4.0.3
Rscript -e 'batchtools::doJobCollection("<%= uri %>")'
exit 0
```
R console:
```r
library(future)
library(future.batchtools)
plan(batchtools_sge, template = "sge_batchtools.tmpl")
f <- future(Sys.sleep(5), resources = list(slots = 2))
# Error: Fatal error occurred: 101. Command 'qsub' produced exit code 2. Output: 'Unable to read script file because of error: ERROR! -pe option must have range as 2nd argument'
```
When I run `debug(BatchtoolsFuture)` on this example, I see that `resources = list(slots = 2)` is not actually passed to the `BatchtoolsFuture()` function. `BatchtoolsFuture()` appears to receive the default `resources = list()`.
## Expected behavior
An SGE job with 2 slots should be submitted.
## Session information
I am using the following commits of `future` and `future.batchtools`:
* https://github.com/HenrikBengtsson/future/commit/344a97f1cc85283095a43fe01cfabf3fb33fdf70
* https://github.com/HenrikBengtsson/future.batchtools/commit/ea3b86cbc6b30f62f78cfd9ef7c5a64f5d42e683
```r
> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux
Matrix products: default
BLAS/LAPACK:
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] future.batchtools_0.10.0-9000 future_1.22.1-9000
loaded via a namespace (and not attached):
[1] parallelly_1.28.1 magrittr_2.0.1 hms_1.1.0 progress_1.2.2
[5] rappdirs_0.3.3 debugme_1.1.0 R6_2.5.1 brew_1.0-6
[9] rlang_0.4.11 fansi_0.5.0 globals_0.14.0 tools_4.0.3
[13] parallel_4.0.3 checkmate_2.0.0 data.table_1.14.0 utf8_1.2.2
[17] withr_2.4.2 ellipsis_0.3.2 base64url_1.4 digest_0.6.27
[21] tibble_3.1.4 lifecycle_1.0.0 crayon_1.4.1 fs_1.5.0
[25] vctrs_0.3.8 batchtools_0.9.15 codetools_0.2-18 stringi_1.7.4
[29] pillar_1.6.2 compiler_4.0.3 backports_1.2.1 prettyunits_1.1.1
[33] listenv_0.8.0 pkgconfig_2.0.3
```
Contributor guide
Assessment
This issue has not been assessed yet.