futureverse / futureverse/future
Only "sequental' plan can work
- Dominant language
- R
- Stars
- 1k
- Forks
- 92
- PR merge metrics
- No merged PRs in 30d
Description
(Please use for Q&A)
**Describe the bug**
I have following code, I have run it in "multicore" with `workers` equal to 1 or 2.
```
future::plan("multicore", workers = 2)
biomisc::run_absolute(
cnv_data[Sample %in% samples],
snv_data[Tumor_Sample_Barcode %in% samples],
primary_disease = "Bladder Cancer",
results_dir = here("results", "ABSOLUTE")
)
```
in the same computer, but run in another R session
```
future::plan("multicore", workers = 1)
biomisc::run_absolute(
cnv_data[Sample %in% samples],
snv_data[Tumor_Sample_Barcode %in% samples],
primary_disease = "Bladder Cancer",
results_dir = here("results", "ABSOLUTE")
)
```
biomisc::run_absolute (https://github.com/Yunuuuu/biomisc/blob/main/R/run_absolute.R) contains following code:
```
p <- progressr::progressor(
along = absolute_filepath[["sample_id"]],
auto_finish = FALSE
)
future.apply::future_lapply(
absolute_filepath[["sample_id"]],
function(sample_id) {
p(type = "update")
maf_fn <- absolute_filepath[["maf"]][[sample_id]]
if (is.na(maf_fn)) {
maf_fn <- NULL
}
absolute_safe(
seg_dat_fn = absolute_filepath[["seg"]][[sample_id]],
maf_fn = maf_fn,
sample_name = sample_id,
sigma_p = sigma_p, max_sigma_h = max_sigma_h,
min_ploidy = min_ploidy, max_ploidy = max_ploidy,
primary_disease = primary_disease, platform = platform,
results_dir = run_absolute_dir,
max_as_seg_count = max_as_seg_count,
max_non_clonal = max_non_clonal,
max_neg_genome = max_neg_genome,
copy_num_type = copy_num_type,
min_mut_af = min_mut_af
)
},
future.globals = TRUE
)
p(type = "done")
```
A clear and concise description of what the bug is.
When I use `workers` 2, I `top` command indicates R usage is around 1-5%.
So I run another R session with `worker` 1, which can lead to a up to 600-650% CPU usage

and after the second R session command returned a result(begin later), the first R session still running here:

**Session information**
R session output:
```
[R]> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.1 LTS
Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/libmkl_rt.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=zh_CN.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=zh_CN.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=zh_CN.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=zh_CN.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] here_1.0.1 data.table_1.14.3
loaded via a namespace (and not attached):
[1] progressr_0.11.0 Rcpp_1.0.9 codetools_0.2-18 listenv_0.8.0 prettyunits_1.1.1 future_1.28.0
[7] crayon_1.5.1 digest_0.6.29 rprojroot_2.0.3 parallelly_1.32.1 R6_2.5.1 lifecycle_1.0.2.9000
[13] future.apply_1.9.1 progress_1.2.2 rlang_1.0.5 cli_3.4.0 vctrs_0.4.1 ellipsis_0.3.2
[19] tools_4.2.1 hms_1.1.2 ABSOLUTE_1.0.6 numDeriv_2016.8-1.1 parallel_4.2.1 compiler_4.2.1
[25] pkgconfig_2.0.3 globals_0.16.1 biomisc_0.0.0.9000
```
the first R future session:
```
[R]> future::futureSessfuture::futureSessionInfo()
*** Package versions
future 1.28.0, parallelly 1.32.1, parallel 4.2.1, globals 0.16.1, listenv 0.8.0
*** Allocations
availableCores():
system nproc
16 16
availableWorkers():
$system
[1] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
[11] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
*** Settings
- future.plan=
- future.fork.multithreading.enable=
- future.globals.maxSize=
- future.globals.onReference=
- future.resolve.recursive=
- future.rng.onMisuse=
- future.wait.timeout=
- future.wait.interval=
- future.wait.alpha=
- future.startup.script=
*** Backends
Number of workers: 2
List of future strategies:
1. multicore:
- args: function (..., workers = 2, envir = parent.frame())
- tweaked: TRUE
- call: future::plan("multicore", workers = 2)
*** Basic tests
Main R session details:
pid r sysname release version nodename machine login user
1 1034146 4.2.1 Linux 5.15.0-47-generic #51-Ubuntu SMP Thu Aug 11 07:51:15 UTC 2022 host001 x86_64 user001 user001
effective_user
1 user001
```
the second future R session
```
[R]> future::futureSes future::futureSessionInfo()
*** Package versions
future 1.28.0, parallelly 1.32.1, parallel 4.2.1, globals 0.16.1, listenv 0.8.0
*** Allocations
availableCores():
system nproc
16 16
availableWorkers():
$system
[1] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
[11] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
*** Settings
- future.plan=
- future.fork.multithreading.enable=
- future.globals.maxSize=
- future.globals.onReference=
- future.resolve.recursive=
- future.rng.onMisuse=
- future.wait.timeout=
- future.wait.interval=
- future.wait.alpha=
- future.startup.script=
*** Backends
Number of workers: 1
List of future strategies:
1. multicore:
- args: function (..., workers = 1, envir = parent.frame())
- tweaked: TRUE
- call: future::plan("multicore", workers = 1)
*** Basic tests
Main R session details:
pid r sysname release version nodename machine login user
1 1036230 4.2.1 Linux 5.15.0-47-generic #51-Ubuntu SMP Thu Aug 11 07:51:15 UTC 2022 host001 x86_64 user001 user001
effective_user
1 user001
Worker R session details:
worker pid r sysname release version nodename machine login user
1 1 1036230 4.2.1 Linux 5.15.0-47-generic #51-Ubuntu SMP Thu Aug 11 07:51:15 UTC 2022 host001 x86_64 user001 user001
effective_user
1 user001
Number of unique worker PIDs: 1 (as expected)
```
Contributor guide
Assessment
This issue has not been assessed yet.