google / google/fuzztest

Have `WithSeeds` accept the `GoogleTest` `ParameterGenerator`s (such as `ValuesIn`, `Combine`, etc).

Open
#46 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
1.1k
Forks
137
Avg merge
3d 6h
Merged PRs (30d)
22

Description

I'm currently in the process of pitching my team on using `FUZZ_TEST` instead of regular old GoogleTest `TEST_P`. Most of our tests currently use `TEST_P`, and my goal is to incrementally convert them from `TEST_P` to `FUZZ_TEST` keeping the behavior the same other than fuzzing. This means keeping the deterministically-executed values the same, which I plan to do with `WithSeeds`.

Unfortunately, `WithSeeds` is less flexible than the [GoogleTest parameter generators](http://google.github.io/googletest/reference/testing.html#INSTANTIATE_TEST_SUITE_P) in a number of important ways:

1. In the case where a `TestWithParam` is parameterized on a single parameter, `TestWithParam` does not insist that that single parameter is wrapped in a `std::tuple`, but `WithSeeds` does.

2. `GoogleTest` provides `Combine`, which allows taking lists of parameter generators and performing the cartesian product of them, producing all possible combinations of the parameter generator's outputs. To replicate this, it looks like I'll either have to manually generate each possible combination or whittle down the inputs and argue to my team that it's ok.

Ideally, I'd like to do neither of these things. I'd like to just continue passing the same parameter generators I sent into `INSTANTIATE_TEST_SUITE_WITH_PARAMS` into `WithSeeds` and have it figure it out. Doing so would make convincing others that I am not introducing regressions much simpler.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.