futureverse / futureverse/future.apply

Distribute different tasks to different types of compute resources

Open
#2 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
R
Stars
218
Forks
20
Avg merge
2d 9h
Merged PRs (30d)
1

Description

Migrated from HenrikBengtsson/future#181, would support wlandau-lilly/drake#169. I would like to have multiple heterogeneous `plans()`'s and distribute different tasks to different `plan()`s.

```r
library(future.batchtools)
plans <- list(plan1 = plan(multicore), plan2 = plan(batchtools_slurm(...)))
which_plans <- rep(c("plan1", "plan2"), each = 5)
future_lapply(
X = 1:10,
FUN = sqrt,
plans = plans,
which_plans = which_plans
)
```

[I tried with ordinary futures outside of `future_lapply()`](https://github.com/wlandau-lilly/drake/issues/169#issuecomment-348503176), but the distributed parallelism on SGE did not work.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.