futureverse / futureverse/future.apply

BUG: Correlated RNG with nested future.apply calls

Open
#108 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
R
Stars
218
Forks
20
Avg merge
2d 9h
Merged PRs (30d)
1

Description

# Issue

```r
library(future.apply)
y <- do.call(rbind, future_lapply(1:3, future.seed = TRUE, FUN = function(i) {
do.call(rbind, future_lapply(1:3, future.seed = TRUE, FUN = function(j) {
data.frame(i = i, j = j, random = runif(n = 1L)) }))
})
)
print(y)
```

gives

```r
i j random
1 1 1 0.8146860
2 1 2 0.4950540
3 1 3 0.9308272
4 2 1 0.4950540
5 2 2 0.9308272
6 2 3 0.2019456
7 3 1 0.9308272
8 3 2 0.2019456
9 3 3 0.6057787
```

Note how some of the random numbers are duplicated, e.g.

```r
> y$random
[1] 0.8146860 0.4950540 0.9308272 0.4950540 0.9308272 0.2019456 0.9308272
[8] 0.2019456 0.6057787
> unique(y$random)
[1] 0.8146860 0.4950540 0.9308272 0.2019456 0.6057787
```

# Troubleshooting

It could be that we've been here before; this seems familiar. I don't have time to investigate in full right now, but it's _not_ that the RNG state of the parent isn't forwarded;

```r
> seed0 <- .Random.seed
> y <- future_lapply(1:3, FUN = function(i) runif(n = 1L), future.seed = TRUE)
> seed <- .Random.seed
> identical(seed, seed0)
[1] FALSE
```

but it could be that it's only forwarded a single step, whereas it needs to be forward `length(X)` steps.

Contributor guide

Open the contributing guide

Research direction

Start by running the nested future_lapply reproducer in the issue and inspect how future.seed is handled across nested calls. The fix is done when the nested results no longer contain duplicated random values while parent and child RNG state behavior remains correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
distributed-systems, hpc
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.