futureverse / futureverse/future

Cannot find symbols exported to node by parallel::clusterExport

Open
#339 6 comments 0 reactions 0 assignees View on GitHub
Backend API feature request feature/sticky-globals globals
Dominant language
R
Stars
1k
Forks
92
PR merge metrics
No merged PRs in 30d

Description

In the following example, I try to export certain variables to each cluster node before any future is created. However, the exported symbols cannot be found when a future is resolved.

I don't want `future()` to detect globals or export variables because in my use case, there are tens of futures and they will be called periodically (every several minutes), each run is time critical so that I don't want the same global variables to be detected and exported to the workers again and again.

```r
library(future)
library(parallel)

test1 <- rnorm(10000)

cl <- makeClusterPSOCK(2)
clusterExport(cl, "test1")
plan(cluster, workers = cl)

f <- future({
sum(test1)
}, globals = FALSE)
values(f)
stopCluster(cl)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.