ChainSafe / ChainSafe/forest

Regression testing using `fuzzy` and the archival data

Open
#3,579 0 comments 0 reactions 0 assignees View on GitHub
CI & Testing Priority: 3 - Medium Ready
Dominant language
Rust
Stars
697
Forks
200
Avg merge
1d 5h
Merged PRs (30d)
65

Description

**Issue summary**

The entire Filecoin blockchain is stored in CloudFlare. With this data, we can do regression testing over arbitrary ranges of epochs.

Regression testing is likely to be too slow for the per-PR CI. Manually triggering the regression tests might be the best option.

The blockchain data covers all of the network versions (currently from NV1 to NV20). Forest has support for NV16 and newer. Epochs lower than `1_960_320` should not be tested.

Tasks:
- [ ] Create a script that automatically downloads snapshots and validates Forest over a range of epochs. The script should accept filters such as network version and epoch range. How many epochs to validate should also be configurable (default to 30k).
- [ ] The script should pick a random block of epochs in the given range and validate using `forest-tool snapshot validate --check-links 0 --check-stateroots EPOCHS`.
- [ ] Make sure files are properly cleaned-up. Otherwise, the server will run out of disk space.
- [ ] Trigger the script to run on `fuzzy` when `@regression-check` is commented in a PR.
- [ ] Support arguments such as `@regression-check nv18` that limit the testing to a specific network version.
- [ ] Write results in a comment response.

**Other information and links**

See Hubert's `grpc_bench` for inspiration w.r.t. workflows triggered by comments: https://github.com/LesnyRumcajs/grpc_bench/blob/master/.github/workflows/issue_bench.yml

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.