Fair sampling with very low sample rates

It may be desirable to run the sampling profiler with a very low sample rate. Instead of sampling every 1ms on 0.1% of users, more representative results could be achieved with one sample every 1s on 100% of users. (Facebook samples stacks from each machine in the server fleet once every 10 seconds to derive a representative dataset.)

However, large sample rates are likely to give a biased sample if not implemented “fairly”. For example, if profiling begins at pageload with a 1s sample rate, you would end up never capturing a sample of any initialization that happens within the first second, leading to misleading, skewed results. This can happen even with lower sample rates, especially if you have interval- or frame-based processes at a regular interval.

However, it’s possible to create a fair sample even with an extremely large sample rate by using an exponential probability distribution: https://en.wikipedia.org/wiki/Exponential_distribution. Each time you take a sample, instead of setting a timer for interval again, you generate a random variate (interval * -ln(Math.random()), I believe) and set a timer for that long. After n * interval has passed, you have still collected on average n samples, but they will have been sampled more fairly instead of at a regular, predictable, (and therefore biased) interval.

Can we consider adding an element to the spec that mandates that browsers use this method, either by default or as opt-in?

Start by reading the js-self-profiling proposal and the discussion on this issue. Evaluate whether exponential sampling should be mandated or offered as an opt-in, and define done as an agreed specification change that addresses fair sampling at low rates.