futureverse / futureverse/future.batchtools

retrieve future expression on local machine

Open
#21 1 comment 0 reactions 0 assignees View on GitHub
feature request question
Dominant language
R
Stars
87
Forks
10
PR merge metrics
No merged PRs in 30d

Description

Hi Henrik,

maybe that's utterly basic and the wrong place to ask, but I thought it might be of interest for more people:

Is there a way you can retrieve a future expressions in the following setup:

```
login <- tweak(remote, workers = "remote@remote")
bsub <- tweak(batchtools_lsf, ...)
plan(list(
login,
bsub,
multiprocess
))
```

and I submit the following job (just an example):

```
y %<-% furrr::future_map_dfr(1:5, function(x){
cores <- availableCores()
furrr::future_map_dfr(1:5, function(x){
tibble::tibble(x = x,
nodename = Sys.info()['nodename'],
Sys.getpid = Sys.getpid(),
availableCores = cores)
})
})
```

... let's imagine this takes an considerable amount of time and I would close my local r session in between submitting the job and its being finished.

After a login then (and rerunning the same plan again) I can't access y, neither can I access y on the remote machine in an R session. Is that possible in general?

If not, we wondered if you can save R objects somehow in an future_apply, submitted from a local machine and evaluated an a remote one?

Sorry for the very basic questions, just started to fiddle around with future.batchtools.

Cheers
Marco

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.