iml complains about future.globals.maxSize even when no future plan is set
- Dominant language
- R
- Stars
- 503
- Forks
- 87
- PR merge metrics
- No merged PRs in 30d
Description
It seems as though {iml} always uses `future.apply` functions even when no `future::plan()` has been set, as you can see here:
```r
library(iml)
library(randomForest)
data("Boston", package = "MASS")
rf <- randomForest(medv ~ ., data = Boston, n.trees = 10)
X <- Boston[which(names(Boston) != "medv")]
predictor <- Predictor$new(rf, data = X, y = Boston$medv)
options(future.globals.maxSize = 1)
FeatureImp$new(predictor, loss = "mape")
# Error in `getGlobalsAndPackages()`:
# ! The total size of the 2 globals exported is 56.59 MiB.. This exceeds the maximum allowed size of 1 bytes (option 'future.globals.maxSize'). There are two globals: ‘FUN’ (28.30 MiB of class ‘function’) and ‘MoreArgs’ (28.29 MiB of class ‘list’)
```
I discovered this after retraining a model whose training data had grown, pushing it just above the default `future.globals.maxSize=500` MB. I was really confused at first because I explicitly chose not to configure a `future::plan` because I have a `predict.function` that handles parallelization.
I think {iml} should have a more explicit way of enabling parallelization to avoid confusion. It would be a better developer experience to not need to read the parallelization vignette in order to debug the default single-threaded behavior of the package.
Contributor guide
Assessment
This issue has not been assessed yet.