Add documentation on simulation testing at scale
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
Now we have open sourced the [Joshua testing framework](https://github.com/FoundationDB/fdb-joshua). It's time to add documentation on how to run simulation testings at scale, probably improve both documentation here and at Joshua project. This will make it easier for contributors other than Apple and Snowflake.
To start, we need to describe the steps for setting up the testing infrastructure:
- Have a FoundationDB cluster running.
- Build and run the docker image from this [script](https://github.com/FoundationDB/fdb-joshua/blob/main/build.sh), make sure the docker image uses the same FDB version as the cluster in the first step. Go inside the docker image and make sure it can talk to the cluster.
- Create a correctness package (i.e., as simple as `ninja package_tests`).
- Submit correctness package to the cluster.
We also need descriptions on how to check simulation results, e.g., how many failed, how to reproduce, inactive jobs, etc.
Contributor guide
Research direction
Start with the existing FoundationDB documentation and Joshua's build.sh, then follow the listed setup flow: a running FoundationDB cluster, a matching Docker image, and a correctness package created with `ninja package_tests`. Document submitting the package and checking simulation results, including failures, reproduction, and inactive jobs, in the relevant documentation for both projects.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- distributed-systems, documentation, testing
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100