NVIDIA / NVIDIA/cudf

[Story] Add fuzz testing to our infrastructure

Open
#16,129 2 comments 0 reactions 0 assignees View on GitHub
feature request pylibcudf Python tests
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
Currently tests of libcudf, pylibcudf, and cuDF Python are a large set of manually written tests. While we endeavor to achieve high coverage rates of the APIs, we inevitably miss data-dependent edge cases, particularly around things like empty data sets.

**Describe the solution you'd like**
We should consider using some fuzz testing library to add more systematic verification of different inputs. I recommend doing this at the Python layer since there is better and simpler tooling available, and because pylibcudf testing can be treated as a superset of libcudf testing in this respect to ensure good coverage of the C++, but we could also choose to do this in C++ if we wish using e.g. [Google's fuzztest](https://github.com/google/fuzztest).

To tackle this problem, we will need to assess the possible options that we have for fuzzing. These include:
- [hypothesis](https://hypothesis.readthedocs.io/en/latest/)
- [ClusterFuzz](github.com/google/clusterfuzz)
- [ClusterFuzzLite](https://github.com/google/clusterfuzzlite)
- [OSS-fuzz](https://github.com/google/oss-fuzz)
- [fuzztest](https://github.com/google/fuzztest)

Once we have evaluated and selected an option, we can move on to implementation questions

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.