futureverse / futureverse/future.apply

using future_apply inside a future

Open
#65 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
R
Stars
218
Forks
20
Avg merge
2d 9h
Merged PRs (30d)
1

Description

since fork is unstable in RStudio, I came to another solution, first using multi-session to create a future from main R session, and use multicore to fork in the future expr body(I've modified future to make supportsMulticore return TRUE from none-interactive R), so future_lapply still use fork

```R
plan(multisession)
res = future({
plan(multicore)
options(mc.cores = 4)
options(future.fork.enable=T)
future_lapply(1:100, function(i){
return(length(BIG_GLOBAL_VAR))
})
})

lapply.res = value(res)

```

I hope such setup should makes big global variable access faster ( should be copied only once) and worker cheaper. with some simple test seems it do complete faster than directly call future_lapply with multisession

my question is : does such setup have some implicit problem ? does future/future.apply expected to work correctly in such setup ? I tried to use such setup with progressr, found that the progressor object can not work, for example

```R
plan(multisession)
with_progress({
p = progressor(100)
future({
plan(multicore)
options(mc.cores = 4)
options(future.fork.enable=T)
future_lapply(1:100, function(i){
p("")
})
})
})
```
is this progressr limitation, or future/future.apply should not be used in such way ? Thanks for advise

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.