RANDOMNESS: L'Cuyer RNG streams instead of ad-hoc random seeds?

Both makeRegistry() and `makeExperimentRegistry() uses:

seed: [integer(1)] Start seed for jobs. Each job uses the (seed + job.id) as seed. Default is a random number in the range [1, .Machine$integer.max/2].

I'm not claiming to be an expert in RNGs, but I believe these type of ad hoc set.seed() approaches is not considered correct by those who do know more about this. Another way to put it is, if it would be this simple, why do RNG methods like the L'Cuyer RNG streams even exist? So, my best guess is that the current approach is likely to produce suboptimal random numbers and in worst case highly correlated random numbers.

I'd like to propose that you look at L'Cuyer RNG streams, which is implemented in the parallel package, meaning you don't need to add extra dependencies. The idea is that your seeds are generated as:

RNGkind("L'Ecuyer-CMRG")
if (!is.null(seed)) set.seed(seed)  ## Set initial seed?
sample.int(n = 1L)  ## Create `.Random.seed` in case missing
seed <- .Random.seed  ## Initial L'Ecuyer-CMRG seed

seeds <- list()
for (kk in seq_along(jobs)) {
  seed <- nextRNGStream(seed)
  seeds[[kk]] <- seed
}

Then export the individual seeds[[job]] values to each job.

To improve on this and allow for nested generation of such seeds, I think one should do something like:

seeds <- list()
for (kk in seq_len(njobs)) {
  seed <- nextRNGStream(seed)
  seeds[[kk]] <- nextRNGSubStream(seed)
}

By using nextRNGSubStream(), my understanding is that process no 3, which receives seed seeds[[3]], can now in turn generate a new stream of seeds from seeds[[3]], without risking generating seeds seeds[4:njobs]. (PS. This is the approach I'm taking in developer's version of the future package.)

If you're not comfortable of making use of L'Cuyer RNG streams right now, may I propose to allow for the option of the user to control this, e.g. seed = myNextSeed where myNextSeed can be a function.

Trace makeRegistry() and makeExperimentRegistry() to find how per-job seeds are created and passed to jobs. Read the parallel package documentation for RNGkind("L'Ecuyer-CMRG"), nextRNGStream(), and nextRNGSubStream(). Done should provide reproducible, non-overlapping streams for jobs, including safe nested stream generation.