[INFRA]: Introduce smoke test mode in CUB benchmarks and tests
- Dominant language
- C++
- Stars
- 2.5k
- Forks
- 486
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 295
Description
### Is this a duplicate?
- [x] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cccl/issues) for this request and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)
### Overview
CUB benchmarks and tests take too long to run (see https://github.com/NVIDIA/cccl/issues/6731). This prevents:
- developers from quickly iterating on changes
- better CI coverage on per-PR actions
To address this, consider having a "smoke test" mode. Instead of covering full combination of types, problem sizes, etc. in hours, this mode would run a quick test of a limited subset of workloads to detect issues in minutes.
As a test example, this radix sort tests runs `8 * 3 = 24` times:
https://github.com/NVIDIA/cccl/blob/f5c350b3e297e0f2ce2033a3b47f8aa57b734bdf/cub/test/catch2_test_device_radix_sort_keys.cu#L81-L82
https://github.com/NVIDIA/cccl/blob/f5c350b3e297e0f2ce2033a3b47f8aa57b734bdf/cub/test/catch2_test_device_radix_sort_keys.cu#L87-L88
In a smoke test mode, one would be enough.
As a benchmark example, this radix sort benchmark runs `7 * 2 * 4 * 3 = 168` combinations:
https://github.com/NVIDIA/cccl/blob/f5c350b3e297e0f2ce2033a3b47f8aa57b734bdf/cub/benchmarks/bench/radix_sort/keys.cu#L60-L64
one per key type, offset type, entropy, and problem size. In a smoke test mode, running only 2^28 elements for 0.544 entropy on one offset type with a couple fundamental types would be enough.
### Details
_No response_
Contributor guide
Research direction
Start with cub/test/catch2_test_device_radix_sort_keys.cu at the linked workload combinations, then compare cub/benchmarks/bench/radix_sort/keys.cu at the referenced combination setup. Trace how CUB tests and benchmarks select workloads and identify an entry point for a limited smoke mode. Done means both examples can run a representative subset in minutes while retaining their normal full coverage mode.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- build-system, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100