google / google/fuzzbench

Feature proposal: FuzzBench aware fuzzers

Open
#831 1 comment 0 reactions 1 assignee Claimed by @jonathanmetzman View on GitHub
Dominant language
Python
Stars
1.2k
Forks
302
PR merge metrics
No merged PRs in 30d

Description

Would be cool to have an API that fuzzers can use to report stats to FuzzBench.
Fuzzers should modify their code to enable more insightful evaluations (like https://hexgolems.com/2020/08/on-measuring-and-visualizing-fuzzer-performance/) like coverage per generated testcase or even execs/sec to evaluate performance.

So FuzzBench will produce two types of experiments, the generic (like now) that works with off-the-shelf fuzzers, and the FuzzBench aware evaluation with only the fuzzers that implements the FuzzBench API.

You can for instance set an env variable like FUZZBENCH_LIB=path/to that defines the path to the C library with such API and then we (fuzzers devs) can adjust the build systems to link such lib and enable the code that does the logging for fuzzbench using the preprocessor.

In AFL, for instance, we can insert snipperts like the following (from https://github.com/google/AFL/blob/master/afl-fuzz.c#L4650):

```c
#ifdef FUZZBENCH_BUILD
#include "fuzzbench.h"
#endif

EXP_ST u8 common_fuzz_stuff(char** argv, u8* out_buf, u32 len) {

u8 fault;

#ifdef FUZZBENCH_BUILD
fuzzbench_log_testcase(out_buf, len);
#endif

...

}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.