API compatibility testing wish list
- Dominant language
- Rust
- Stars
- 697
- Forks
- 200
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 65
Description
**Issue summary**
Assorted features that would be nice to have:
- [x] Filtering based on method name. Similar to how tests can be filtered when running `cargo nextest` or `cargo test`.
- [ ] Option for generating exhaustive tests based on snapshots. Right now, we only tests a small subset of the possible queries. With exhaustive testing, we would generate a `Filecoin.ChainGetTipSetByHeight` call for each available epoch (calibnet is currently at around 1 million epochs). Same goes for `Filecoin.ChainGetBlock`, `Filecoin.ChainGetMessage`, `Filecoin.StateGetActor`, `Filecoin.StateMinerPower`, etc. These calls have a finite set of input arguments and we can exhaustively test all of them.
- [ ] Option for sampling tests. Running millions or tens of millions of tests is time consuming. It would be nice to run, say, 100 randomly sampled tests for each RPC method. This is similar to what quickcheck does.
- [ ] Run tests in parallel.
**Other information and links**
Contributor guide
Assessment
This issue has not been assessed yet.