[FEA] Reduce benchmark search space where applicable
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Significantly lower the variations of parameters for the long-running benchmarks.**
Quote from https://github.com/rapidsai/cudf/issues/5773#issue-666438739
> Type Dispatcher, for example, has too many parameter variations. One parameter range goes from 1024 --> 67 million going up by a power of two each interval. This can easily be cut down by doing a power of 4. This approach can be applied to many of our biggest offenders in runtime.
**Reduce the number of benchmarks that have an iteration count of <= 2**
We have benchmarks that are run a single time, which provides insufficient iterations to trust the results.
If valid for smaller data sizes, they should be be moved to do so.
If they need to run on larger datasize we need to switch them to NVBench, and state they need longer runtimes
Contributor guide
Assessment
This issue has not been assessed yet.