futureverse / futureverse/future

About the `future.seed` default

Open
#365 1 comment 0 reactions 0 assignees View on GitHub
RNG
Dominant language
R
Stars
1k
Forks
92
PR merge metrics
No merged PRs in 30d

Description

Quoting from our mail conversation

> The current default with future.seed=FALSE is not set in stone but I need to think about it more. Ideally, if we could detect whether parallel RNG is needed or not, it could be set automatically. But I doubt that will ever be possible - it would require annotating all functions specify if they use RNGs or not. There was also the discussion of detecting when RNGs were indeed used even if future.seed=FALSE. If detected, a warning or even an error could be produced. This would not be too hard to implement and the overhead would be minimal. This should prevent calling future_lapply() et al. without future.seed=TRUE when truly
needed. (This is on my radar since a while)

Regarding a dynamic setting of `future.seed`: Would it make sense to check which future::plan is requested? If it is a parallel one, turn it on internally by default - if not, leave it off.

This way users would use the default RNG kind when using `plan(sequential)` and the "L'Ecuyer-CMRG" one in parallel scenarios.
Due to `future.seed = TRUE`, both would magically work with just `set.seed()` and there is no overhead when it is not needed.

When it comes to scenarios when reproducibility is not wanted but only speed: It would be great if users could turn `future.seed` off on their side and not rely on what a package devs set it to within the package.
Hence I'd like to an option to overwrite `future.seed` on the user level when setting the future::plan() - this is even unrelated to all other ideas in here.

With all the options from above, practical scenarios could look as follows:

- Parallel processes are reproducible by default because "L'Ecuyer-CMRG" is used via a dynamic `future.seed` argument
- No overhead for sequential runs (`future.seed = FALSE` always). If a sequential plan detected `future.seed = TRUE`, a warning could be issued
- If speed is > reproducibility, users can turn off the latter by setting `future::plan(, future.seed = FALSE)` which will take precedence over any settings downstream in any `future_*apply()` call

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.