google / google/fuzzbench

Archive and release private experiments

Open
#1,096 6 comments 1 reaction 0 assignees View on GitHub
question
Dominant language
Python
Stars
1.2k
Forks
302
PR merge metrics
No merged PRs in 30d

Description

Many academic papers user fuzzbench to evaluate their contributions vs other fuzzers.
Rarely (as in "I have never seen this done but it might have happened without me having noticed") however the testing setup and the Dockerfile of the new fuzzer is made available.
This makes assessing and reassessing papers impossible.

E.g. getting symqemu running is so far failed by Jonathan and me individually. In other experiments - did e.g. afl++ in the experiment have cmplog enabled and which version was it tested against? Impossible to know.

I therefore propose to archive private experiments and make these available 6 months later - unless the requestor actively announces this should not be done.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.