NVIDIA / NVIDIA/cccl

Implement scripts for detection of performance regressions across CUB versions

Open
#926 0 comments 3 reactions 0 assignees View on GitHub
cub
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

Tuning infrastructure is currently used to find better tuning variants. Reusing it for the detection of performance regressions is preferable since there should be a single source of truth for workloads and benchmarks we care about. The detection of performance regressions should be based on sample distribution analysis. We can start with Mann–Whitney U test, and if it's likely that two distributions do not belong to a single one, proceed with the ratio function based on the Harrell-Davis quantile estimator. It might also be worth it to experiment with outlier detection.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.