google / google/fuzztest

Indicate Domains that are cheap vs. expensive to vary, and vary the cheap ones first

Open
#634 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
1.1k
Forks
137
Avg merge
3d 6h
Merged PRs (30d)
22

Description

Is there some way we can indicate that some Domains are expensive to change, while others are cheap and should be varied first? For example, imagine a situation where the code under test can execute much faster if the expensive Domains are unchanged while the cheap Domains are varied arbitrarily. Once the cheap Domains have been 'exhausted' of interesting things by varying them, the expensive Domains are allowed to arbitrarily change exactly once. Then we go back to varying the cheap Domains, rinse and repeat.

A prime example of this would be testing a compiler and the generated program. Assume compilation is expensive while executing the generated program on a specific program input is cheap. The Domains that change the compiler's behavior should be varied less frequently, and only when the generated program has been thoroughly tested by varying all the Domains that are inputs to the generated program. The code under test would, of course, only recompile the program when needed (i.e. when the inputs defining the program to compile have changed), effectively caching the compiled program.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.