apple / apple/foundationdb

Add documentation on simulation testing at scale

Open
#4,663 1 comment 0 reactions 0 assignees View on GitHub
documentation good first issue
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

Now we have open sourced the [Joshua testing framework](https://github.com/FoundationDB/fdb-joshua). It's time to add documentation on how to run simulation testings at scale, probably improve both documentation here and at Joshua project. This will make it easier for contributors other than Apple and Snowflake.

To start, we need to describe the steps for setting up the testing infrastructure:

- Have a FoundationDB cluster running.
- Build and run the docker image from this [script](https://github.com/FoundationDB/fdb-joshua/blob/main/build.sh), make sure the docker image uses the same FDB version as the cluster in the first step. Go inside the docker image and make sure it can talk to the cluster.
- Create a correctness package (i.e., as simple as `ninja package_tests`).
- Submit correctness package to the cluster.

We also need descriptions on how to check simulation results, e.g., how many failed, how to reproduce, inactive jobs, etc.

Contributor guide

Open the contributing guide

Research direction

Start with the existing FoundationDB documentation and Joshua's build.sh, then follow the listed setup flow: a running FoundationDB cluster, a matching Docker image, and a correctness package created with `ninja package_tests`. Document submitting the package and checking simulation results, including failures, reproduction, and inactive jobs, in the relevant documentation for both projects.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
distributed-systems, documentation, testing
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.