[FEA]: Tuning Verification
- Dominant language
- C++
- Stars
- 2.5k
- Forks
- 486
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 295
Description
### Is this a duplicate?
- [x] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cccl/issues) for this request and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)
### Area
General CCCL
### Is your feature request related to a problem? Please describe.
CCCL unifies benchmarks and tunings under a single file, for instance:
https://github.com/NVIDIA/cccl/blob/5ea5d42567693aad8c0e5d6316155ecc612fdc71/cub/benchmarks/bench/adjacent_difference/subtract_left.cu#L45-L71
This file produces two executables:
- `cub.bench.adjacent_difference.subtract_left.base` - tuning baseline `TUNE_BASE=1`
- `cub.bench.adjacent_difference.subtract_left.variant` - tuning variant `TUNE_BASE=0`
Base version is used in performance CI and as performance reference for tuning work. Tuning version is used to tweak tuning parameters and algorithm implementations (manually or automatically). As we change the implementation, we don't know if the performance data we see is coming from a correct algorithm implementation until we run the unit tests.
This leads to a couple of issues:
- During manual performance tuning, building test and benchmark increases turnaround time.
- During automated performance tuning, incorrect search paths are not terminated early.
### Describe the solution you'd like
We should verify if the algorithm produces correct output in tuning mode `TUNE_BASE=0` and adopt a practice of running `*.variant` executables when working on algorithm optimizations.
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.