NVIDIA / NVIDIA/cccl

[FEA]: Tuning Verification

Open
#9,308 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

### Is this a duplicate?

- [x] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cccl/issues) for this request and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)

### Area

General CCCL

### Is your feature request related to a problem? Please describe.

CCCL unifies benchmarks and tunings under a single file, for instance:

https://github.com/NVIDIA/cccl/blob/5ea5d42567693aad8c0e5d6316155ecc612fdc71/cub/benchmarks/bench/adjacent_difference/subtract_left.cu#L45-L71

This file produces two executables:

- `cub.bench.adjacent_difference.subtract_left.base` - tuning baseline `TUNE_BASE=1`
- `cub.bench.adjacent_difference.subtract_left.variant` - tuning variant `TUNE_BASE=0`

Base version is used in performance CI and as performance reference for tuning work. Tuning version is used to tweak tuning parameters and algorithm implementations (manually or automatically). As we change the implementation, we don't know if the performance data we see is coming from a correct algorithm implementation until we run the unit tests.

This leads to a couple of issues:
- During manual performance tuning, building test and benchmark increases turnaround time.
- During automated performance tuning, incorrect search paths are not terminated early.

### Describe the solution you'd like

We should verify if the algorithm produces correct output in tuning mode `TUNE_BASE=0` and adopt a practice of running `*.variant` executables when working on algorithm optimizations.

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.