NVIDIA / NVIDIA/cccl

Some benchmarks take really long

Open
#6,731 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

Running the entire CUB+Thrust benchmarks using `run.py` takes >4h on a B200 and thus requires at least 2 batch jobs. We should reduce this time a bit, since it also helps with producing regression benchmarks more quickly.

Here are the top offenders, where each compile-time workload took longer than 100s to run:
```
cub.bench.segmented_radix_sort.keys.base
cub.bench.segmented_sort.keys.base
cub.bench.topk.keys.base
cub.bench.topk.pairs.base
```
Full table: https://docs.google.com/spreadsheets/d/131YcBo2XpZXVn8CE7OAdh16cd16Ui6CfgL05V-BWrf4/edit?usp=sharing

The cumulative runtime of all segmented_radix_sort benchmarks is for example 136 minutes.

Sadly, the database does not keep around which compile-time workloads took the longest.

I think we should reduce the number of variants for those benchmarks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.