futureverse / futureverse/future.apply
BUG: Correlated RNG with nested future.apply calls
- Dominant language
- R
- Stars
- 218
- Forks
- 20
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 1
Description
# Issue
```r
library(future.apply)
y <- do.call(rbind, future_lapply(1:3, future.seed = TRUE, FUN = function(i) {
do.call(rbind, future_lapply(1:3, future.seed = TRUE, FUN = function(j) {
data.frame(i = i, j = j, random = runif(n = 1L)) }))
})
)
print(y)
```
gives
```r
i j random
1 1 1 0.8146860
2 1 2 0.4950540
3 1 3 0.9308272
4 2 1 0.4950540
5 2 2 0.9308272
6 2 3 0.2019456
7 3 1 0.9308272
8 3 2 0.2019456
9 3 3 0.6057787
```
Note how some of the random numbers are duplicated, e.g.
```r
> y$random
[1] 0.8146860 0.4950540 0.9308272 0.4950540 0.9308272 0.2019456 0.9308272
[8] 0.2019456 0.6057787
> unique(y$random)
[1] 0.8146860 0.4950540 0.9308272 0.2019456 0.6057787
```
# Troubleshooting
It could be that we've been here before; this seems familiar. I don't have time to investigate in full right now, but it's _not_ that the RNG state of the parent isn't forwarded;
```r
> seed0 <- .Random.seed
> y <- future_lapply(1:3, FUN = function(i) runif(n = 1L), future.seed = TRUE)
> seed <- .Random.seed
> identical(seed, seed0)
[1] FALSE
```
but it could be that it's only forwarded a single step, whereas it needs to be forward `length(X)` steps.
Contributor guide
Research direction
Start by running the nested future_lapply reproducer in the issue and inspect how future.seed is handled across nested calls. The fix is done when the nested results no longer contain duplicated random values while parent and child RNG state behavior remains correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- distributed-systems, hpc
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100