NVIDIA / NVIDIA/cccl

[INFRA]: Introduce smoke test mode in CUB benchmarks and tests

Open
#9,569 0 comments 0 reactions 0 assignees View on GitHub
infra
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

### Is this a duplicate?

- [x] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cccl/issues) for this request and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)

### Overview

CUB benchmarks and tests take too long to run (see https://github.com/NVIDIA/cccl/issues/6731). This prevents:
- developers from quickly iterating on changes
- better CI coverage on per-PR actions

To address this, consider having a "smoke test" mode. Instead of covering full combination of types, problem sizes, etc. in hours, this mode would run a quick test of a limited subset of workloads to detect issues in minutes.

As a test example, this radix sort tests runs `8 * 3 = 24` times:

https://github.com/NVIDIA/cccl/blob/f5c350b3e297e0f2ce2033a3b47f8aa57b734bdf/cub/test/catch2_test_device_radix_sort_keys.cu#L81-L82

https://github.com/NVIDIA/cccl/blob/f5c350b3e297e0f2ce2033a3b47f8aa57b734bdf/cub/test/catch2_test_device_radix_sort_keys.cu#L87-L88

In a smoke test mode, one would be enough.

As a benchmark example, this radix sort benchmark runs `7 * 2 * 4 * 3 = 168` combinations:

https://github.com/NVIDIA/cccl/blob/f5c350b3e297e0f2ce2033a3b47f8aa57b734bdf/cub/benchmarks/bench/radix_sort/keys.cu#L60-L64

one per key type, offset type, entropy, and problem size. In a smoke test mode, running only 2^28 elements for 0.544 entropy on one offset type with a couple fundamental types would be enough.

### Details

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with cub/test/catch2_test_device_radix_sort_keys.cu at the linked workload combinations, then compare cub/benchmarks/bench/radix_sort/keys.cu at the referenced combination setup. Trace how CUB tests and benchmarks select workloads and identify an entry point for a limited smoke mode. Done means both examples can run a representative subset in minutes while retaining their normal full coverage mode.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
build-system, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.