KhronosGroup / KhronosGroup/SPIRV-Tools

spirv-fuzz: centralize probabilities so we can easily bias random choices

Open
#2,849 0 comments 0 reactions 0 assignees View on GitHub
component:fuzzer
Dominant language
C++
Stars
1.4k
Forks
709
Avg merge
1d 22h
Merged PRs (30d)
28

Description

For example, the following allows the caller to specify probabilities:

`rng->ChoosePercentage(GetChanceOfMovingBlockDown())`

Instead, the rng caller should provide the number of choices it has available, plus some hints as to why it needs to make the choice and any other information that could be useful:

```c++
rng->ChooseIntUpTo(
15, // the number of ways to fuzz a constant
// (would probably be something like WaysToFuzz.SIZE, assuming this was an enum)
CHOOSING_HOW_TO_FUZZ_CONSTANT, // a hint at why we are making this choice
{DEPTH_HINT, 5} // further pairs adding extra, optional information, that won't necessarily be used or applicable
)
```

Thus, the implementation of `ChooseIntUpTo` can always just return a uniform random choice. But, we can easily add additional checks as needed (a switch statement) to bias probabilities from a centralized place and even tweak these at run time. Furthermore, callers can easily add as much additional information as they like, even if it won't be used yet.

When the implementation sees `CHOOSING_HOW_TO_FUZZ_CONSTANT`, it may know that the choice is being made over all enum values of `WaysToFuzz`. This won't work if the number of choices varies depending on what is possible in the given context; in this case, perhaps the meaning of each choice should be clarified via hints, or perhaps we should have another `Choose*` method that takes the list of choices (as an iterable list of size_t, assuming they will always be enums).

```c++
rng->ChooseEnum(
ways_to_fuzz, // vector of enum values; possibly need to cast it or could
// template this function based on enum type
CHOOSING_HOW_TO_FUZZ_CONSTANT, // a hint at why we are making this choice
{DEPTH_HINT, 5} // further pairs adding extra, optional information, that won't necessarily be used or applicable
)
```

The downside to templating is perhaps it would add more effort if we just want to, at run time, decide to switch to code that doesn't distinguish between the type of the enum. Not completely sure how this would look though.

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing ChoosePercentage and GetChanceOfMovingBlockDown entry points in spirv-fuzz and trace how callers currently select random choices. Define the centralized choice API, including enum and hint handling, then verify that probability biasing can be adjusted centrally without losing context-dependent choices.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.