futureverse / futureverse/globals

Formulas don't seem to be searched for globals

Open
#87 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
R
Stars
29
Forks
3
PR merge metrics
No merged PRs in 30d

Description

First seen here https://github.com/DavisVaughan/furrr/issues/256

I've reduced that down to this minimal ish example. It seems like formula objects aren't searched in for globals? I'm not quite sure. If you do decide to look in formula objects for globals, it will be important to ensure that you look in the formula's environment for those globals rather than the standard `envir` argument.

The only way I found around this was to wrap up the formula creation in `local()` so the `constant` is contained in an environment that gets shipped along to the worker alongside the formula.

```r
library(future)

set.seed(123)

plan(multisession, workers = 2)

df <- data.frame(
y = sample(10),
x = sample(10)
)

constant <- rep(0, nrow(df))
formula <- y ~ x + constant

# This works
model.matrix(formula, data = df)
#> (Intercept) x constant
#> 1 1 10 0
#> 2 1 5 0
#> 3 1 3 0
#> 4 1 8 0
#> 5 1 1 0
#> 6 1 4 0
#> 7 1 6 0
#> 8 1 9 0
#> 9 1 7 0
#> 10 1 2 0
#> attr(,"assign")
#> [1] 0 1 2

# This doesn't
result <- future::future({
model.matrix(formula, data = df)
})

# Uh oh
value(result)
#> Error in eval(predvars, data, env): object 'constant' not found

# //////////////////////////////////////////////////////////////////////////////

# globals is not seeing `constant`
globals::globalsOf(quote({
model.matrix(formula, data = df)
}))
#> $`{`
#> .Primitive("{")
#>
#> $model.matrix
#> function (object, ...)
#> UseMethod("model.matrix")
#>
#>
#>
#> $formula
#> y ~ x + constant
#>
#> $df
#> y x
#> 1 3 10
#> 2 10 5
#> 3 2 3
#> 4 8 8
#> 5 6 1
#> 6 9 4
#> 7 1 6
#> 8 7 9
#> 9 5 7
#> 10 4 2
#>
#> attr(,"where")
#> attr(,"where")$`{`
#>
#>
#> attr(,"where")$model.matrix
#>
#> attr(,"name")
#> [1] "package:stats"
#> attr(,"path")
#> [1] "/Library/Frameworks/R.framework/Versions/4.2/Resources/library/stats"
#>
#> attr(,"where")$formula
#>
#>
#> attr(,"where")$df
#>
#>
#> attr(,"class")
#> [1] "Globals" "list"

# Note that the environment of `formula` is the global env.
# Since the global env isn't serialized to the worker, the `constant` won't
# be available over on the worker

# //////////////////////////////////////////////////////////////////////////////

# Here is a trick that does work, taking advantage of the fact that a `formula`
# keeps track of its environment

formula <- local({
constant <- rep(0, nrow(df))
y ~ x + constant
})

# Note the env here isn't the global env
formula
#> y ~ x + constant
#>

# So now this works
result <- future::future({
model.matrix(formula, data = df)
})

value(result)
#> (Intercept) x constant
#> 1 1 10 0
#> 2 1 5 0
#> 3 1 3 0
#> 4 1 8 0
#> 5 1 1 0
#> 6 1 4 0
#> 7 1 6 0
#> 8 1 9 0
#> 9 1 7 0
#> 10 1 2 0
#> attr(,"assign")
#> [1] 0 1 2
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.