etcd-io / etcd-io/etcd

Antithesis tests should use /dev/random rather than the math/rand package

Open
#21,720 10 comments 0 reactions 1 assignee Assigned to @nwnt View on GitHub
area/robustness-testing type/feature
Dominant language
Go
Stars
52.3k
Forks
10.5k
Avg merge
2d 21h
Merged PRs (30d)
43

Description

### What would you like to be added?

In the Antithesis robustness tests, randomness should be acquired by reading from `/dev/random` directly, rather than wrapping Go's `math/rand`.

Specifically, when run in Antithesis the functions in [this file](https://github.com/etcd-io/etcd/blob/0cbc031e7f224102bd1ec42a81f659a2b8c3350c/tests/robustness/random/random.go#L18) should read fresh bytes from `/dev/random` on every invocation, and transform them into the required value range. The same goes for any application code that uses randomness (e.g. to randomize backoff durations).

Antithesis also provides a Go SDK which does this wrapping in a more convenient form: https://antithesis.com/docs/generated/sdk/golang/random/

### Why is this needed?

Hey, I work at Antithesis. I was reading through the `tests/antithesis` directory the other day and noticed a problem with the way you're generating random values.

When you import Go's `math/rand`, it initializes a PRNG seeded from kernel entropy. This seed remains fixed thereafter, for the whole lifetime of the program.

Antithesis controls all sources of entropy in the deterministic simulation. This includes the bytes read from the random devices, and also the OS clock and performance counters. The fuzzer attempts to trigger different behaviors by branching the timeline at different moments and providing different random values.

But if a program uses a PRNG with a fixed seed, the fuzzer cannot directly affect its behavior beyond the initial seed choice. The random draws are "opaque" to the fuzzer. Every timeline that branches after the program begins will observe an **identical** sequence of random values from the PRNG. The only way the random sequences can diverge is when the program restarts (e.g. from a crash or a kill signal) and chooses a new seed.

If instead you always acquire fresh bytes from `/dev/random`, every random choice becomes visible to the fuzzer, becoming a potential divergence point for new behaviors. This will likely increase the power of the fuzz tests.

More info in the docs: https://antithesis.com/docs/environment/the_antithesis_environment/?sid=8fce.35&sterm=random&marks=Random#random-devices

(Putting this as a feature request since this arguably isn't a "bug", just the tests aren't as efficient as they could be.)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.