ChainSafe / ChainSafe/forest

API compatibility testing wish list

Open
#3,641 0 comments 0 reactions 0 assignees View on GitHub
Priority: 4 - Low Type: Enhancement
Dominant language
Rust
Stars
697
Forks
200
Avg merge
1d 5h
Merged PRs (30d)
65

Description

**Issue summary**

Assorted features that would be nice to have:

- [x] Filtering based on method name. Similar to how tests can be filtered when running `cargo nextest` or `cargo test`.
- [ ] Option for generating exhaustive tests based on snapshots. Right now, we only tests a small subset of the possible queries. With exhaustive testing, we would generate a `Filecoin.ChainGetTipSetByHeight` call for each available epoch (calibnet is currently at around 1 million epochs). Same goes for `Filecoin.ChainGetBlock`, `Filecoin.ChainGetMessage`, `Filecoin.StateGetActor`, `Filecoin.StateMinerPower`, etc. These calls have a finite set of input arguments and we can exhaustively test all of them.
- [ ] Option for sampling tests. Running millions or tens of millions of tests is time consuming. It would be nice to run, say, 100 randomly sampled tests for each RPC method. This is similar to what quickcheck does.
- [ ] Run tests in parallel.

**Other information and links**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.