Cross benchmark pairwise differential coverage plot
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 302
- PR merge metrics
- No merged PRs in 30d
Description
It would be useful too have a visualization showing which fuzzers complement each other well in general, across benchmarks.
Not clear, but an aggregate / cross-benchmark version of the current per-benchmark pairwise unique coverage heatmap plots *might* help with this. While the current per-benchmark plots show the number of edges covered by fuzzer 1 but not by fuzzer 2, we could compute the aggregate matrix plot as follows:
- for each benchmark, and for each fuzzer pair (f1, f2), compute the ratio of the unique and combined coverage,
ie. ratio_{bench,f1,f2} = C(f1)\C(f2) / C(f1)UC(f2).
- then for the aggregate matrix, for each fuzzer pair (f1, f2), take the average of the per-benchmark ratios
ie. agg_ratio_{f1,f2} = AVG_{bench in all benchmarks}(ratio_{benc,f1,f2})
Prior discussion: https://github.com/google/fuzzbench/issues/719#issuecomment-692940683
Contributor guide
Assessment
This issue has not been assessed yet.