NVIDIA / NVIDIA/cccl

Tune `DeviceSelect::Unique for H100, B200 for types for which performance regressed after #5506

Open
#5,597 1 comment 0 reactions 1 assignee Claimed by @gonidelis View on GitHub
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

After the bug fix in https://github.com/NVIDIA/cccl/pull/5566, we saw some performance regressions of `>5%` in some workloads, particularly for `i32` and `f32` types (so far only benchmarked on H100).

**Important Update:**
In https://github.com/NVIDIA/cccl/pull/5566, we ultimately settled for a solution that does one of two things to avoid invoking equality operator on unexpected values:
1. for primitive(!) types AND ::cuda::std::equal_to: we compare all items, including out-of-bounds items, and later correct the flags of the out-of-bounds items (this is very similar to old code path).
2. otherwise: we avoid invoking the equality operator on any out-of-bounds items
For workloads that meet the criteria from (1), we can avoid any performance regressions

So, for (1) we do not see performance regressions anymore. All our CUB benchmark workloads fall under the category of (1). We yet need to benchmark the performance downside of the new code path from (2).

- [ ] Add a benchmark workload for code path (2).
- [ ] Benchmark performance difference for before and after https://github.com/NVIDIA/cccl/pull/5506
- [ ] Re-tune for the types for which performance regressed on H100
- [ ] Re-tune for the types for which performance regressed on B200

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.