Tune `DeviceSelect::Unique for H100, B200 for types for which performance regressed after #5506
- Dominant language
- C++
- Stars
- 2.5k
- Forks
- 486
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 295
Description
After the bug fix in https://github.com/NVIDIA/cccl/pull/5566, we saw some performance regressions of `>5%` in some workloads, particularly for `i32` and `f32` types (so far only benchmarked on H100).
**Important Update:**
In https://github.com/NVIDIA/cccl/pull/5566, we ultimately settled for a solution that does one of two things to avoid invoking equality operator on unexpected values:
1. for primitive(!) types AND ::cuda::std::equal_to: we compare all items, including out-of-bounds items, and later correct the flags of the out-of-bounds items (this is very similar to old code path).
2. otherwise: we avoid invoking the equality operator on any out-of-bounds items
For workloads that meet the criteria from (1), we can avoid any performance regressions
So, for (1) we do not see performance regressions anymore. All our CUB benchmark workloads fall under the category of (1). We yet need to benchmark the performance downside of the new code path from (2).
- [ ] Add a benchmark workload for code path (2).
- [ ] Benchmark performance difference for before and after https://github.com/NVIDIA/cccl/pull/5506
- [ ] Re-tune for the types for which performance regressed on H100
- [ ] Re-tune for the types for which performance regressed on B200
Contributor guide
Assessment
This issue has not been assessed yet.