Tune histogram range for H100/32bit types
- Dominant language
- C++
- Stars
- 2.5k
- Forks
- 486
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 295
Description
The following [benchmark](https://github.com/senior-zero/thrust_benchmark/blob/main/benches/cub/device/histogram/range.cu) illustrates some issues of histogram range tuning on H100. Currently, histogram range performs better on A100 than on H100 when used with 32bit types (and curtain `BinsCount`:
```
| T | BinsCount | Pattern | Elements | Ref Time | Ref Noise | Cmp Time | Cmp Noise | Diff | %Diff | Status |
|-----|-------------|-----------|------------|------------|-------------|------------|-------------|--------------|---------|----------|
| U32 | 2048 | Seq | 2^22 | 78.140 us | 1.72% | 133.233 us | 2.48% | 55.092 us | 70.50% | FAIL |
| U32 | 2048 | Seq | 2^24 | 236.257 us | 0.34% | 468.364 us | 1.20% | 232.107 us | 98.24% | FAIL |
| U32 | 2048 | Seq | 2^26 | 852.727 us | 0.19% | 1.776 ms | 0.74% | 923.267 us | 108.27% | FAIL |
| U32 | 2048 | Seq | 2^28 | 3.302 ms | 0.05% | 7.035 ms | 0.54% | 3.733 ms | 113.07% | FAIL |
| U32 | 2048 | ModSeq | 2^22 | 190.964 us | 0.65% | 212.733 us | 1.75% | 21.769 us | 11.40% | FAIL |
| U32 | 2048 | ModSeq | 2^24 | 601.287 us | 0.78% | 709.520 us | 1.22% | 108.232 us | 18.00% | FAIL |
| U32 | 2048 | ModSeq | 2^26 | 1.958 ms | 0.86% | 2.671 ms | 0.99% | 713.390 us | 36.44% | FAIL |
| U32 | 2048 | ModSeq | 2^28 | 7.184 ms | 0.43% | 10.546 ms | 0.97% | 3.362 ms | 46.79% | FAIL |
| U32 | 2048 | Const | 2^22 | 82.140 us | 0.83% | 146.105 us | 2.50% | 63.965 us | 77.87% | FAIL |
| U32 | 2048 | Const | 2^24 | 250.787 us | 0.27% | 507.239 us | 1.56% | 256.452 us | 102.26% | FAIL |
| U32 | 2048 | Const | 2^26 | 912.483 us | 0.08% | 1.953 ms | 1.92% | 1.040 ms | 114.02% | FAIL |
| U32 | 2048 | Const | 2^28 | 3.559 ms | 0.04% | 7.736 ms | 1.69% | 4.177 ms | 117.36% | FAIL |
| U32 | 2048 | Rand | 2^22 | 83.411 us | 0.84% | 143.699 us | 2.33% | 60.288 us | 72.28% | FAIL |
| U32 | 2048 | Rand | 2^24 | 253.641 us | 0.27% | 503.410 us | 0.68% | 249.769 us | 98.47% | FAIL |
| U32 | 2048 | Rand | 2^26 | 936.891 us | 0.08% | 1.929 ms | 0.53% | 992.330 us | 105.92% | FAIL |
| U32 | 2048 | Rand | 2^28 | 3.827 ms | 0.65% | 7.755 ms | 0.51% | 3.928 ms | 102.62% | FAIL |
| F32 | 2048 | Seq | 2^22 | 79.306 us | 0.97% | 135.848 us | 2.48% | 56.542 us | 71.30% | FAIL |
| F32 | 2048 | Seq | 2^24 | 239.548 us | 0.34% | 468.652 us | 1.16% | 229.104 us | 95.64% | FAIL |
| F32 | 2048 | Seq | 2^26 | 864.102 us | 0.18% | 1.785 ms | 0.75% | 920.722 us | 106.55% | FAIL |
| F32 | 2048 | Seq | 2^28 | 3.346 ms | 0.05% | 7.061 ms | 0.55% | 3.714 ms | 110.99% | FAIL |
| F32 | 2048 | ModSeq | 2^22 | 192.238 us | 0.64% | 215.052 us | 1.72% | 22.814 us | 11.87% | FAIL |
| F32 | 2048 | ModSeq | 2^24 | 605.508 us | 0.72% | 710.501 us | 1.18% | 104.994 us | 17.34% | FAIL |
| F32 | 2048 | ModSeq | 2^26 | 1.964 ms | 0.85% | 2.675 ms | 0.99% | 710.148 us | 36.15% | FAIL |
| F32 | 2048 | ModSeq | 2^28 | 7.208 ms | 0.50% | 10.550 ms | 0.93% | 3.341 ms | 46.35% | FAIL |
| F32 | 2048 | Const | 2^22 | 83.576 us | 0.83% | 145.308 us | 2.52% | 61.733 us | 73.86% | FAIL |
| F32 | 2048 | Const | 2^24 | 253.968 us | 0.27% | 510.663 us | 1.59% | 256.696 us | 101.07% | FAIL |
| F32 | 2048 | Const | 2^26 | 923.552 us | 0.08% | 1.960 ms | 1.99% | 1.036 ms | 112.21% | FAIL |
| F32 | 2048 | Const | 2^28 | 3.601 ms | 0.03% | 7.772 ms | 1.76% | 4.171 ms | 115.82% | FAIL |
| F32 | 2048 | Rand | 2^22 | 78.341 us | 0.82% | 131.097 us | 2.49% | 52.755 us | 67.34% | FAIL |
| F32 | 2048 | Rand | 2^24 | 235.690 us | 0.32% | 456.201 us | 0.60% | 220.511 us | 93.56% | FAIL |
| F32 | 2048 | Rand | 2^26 | 853.182 us | 0.31% | 1.752 ms | 0.58% | 899.208 us | 105.39% | FAIL |
| F32 | 2048 | Rand | 2^28 | 3.330 ms | 0.04% | 6.898 ms | 0.23% | 3.567 ms | 107.12% | FAIL |
```
For 64bit types there's expected improvement:
```
| T | BinsCount | Pattern | Elements | Ref Time | Ref Noise | Cmp Time | Cmp Noise | Diff | %Diff | Status |
|-----|-------------|-----------|------------|------------|-------------|------------|-------------|--------------|---------|----------|
| U64 | 2048 | Seq | 2^22 | 88.463 us | 2.57% | 64.517 us | 2.12% | -23.946 us | -27.07% | FAIL |
| U64 | 2048 | Seq | 2^24 | 249.174 us | 0.71% | 170.488 us | 1.17% | -78.687 us | -31.58% | FAIL |
| U64 | 2048 | Seq | 2^26 | 903.151 us | 0.26% | 552.999 us | 0.51% | -350.153 us | -38.77% | FAIL |
| U64 | 2048 | Seq | 2^28 | 3.492 ms | 0.47% | 2.031 ms | 0.16% | -1460.634 us | -41.83% | FAIL |
| U64 | 2048 | ModSeq | 2^22 | 132.497 us | 1.58% | 103.127 us | 1.39% | -29.370 us | -22.17% | FAIL |
| U64 | 2048 | ModSeq | 2^24 | 397.852 us | 0.90% | 295.433 us | 1.05% | -102.419 us | -25.74% | FAIL |
| U64 | 2048 | ModSeq | 2^26 | 1.475 ms | 0.51% | 1.056 ms | 0.69% | -419.501 us | -28.44% | FAIL |
| U64 | 2048 | ModSeq | 2^28 | 5.781 ms | 0.34% | 4.112 ms | 0.63% | -1668.993 us | -28.87% | FAIL |
| U64 | 2048 | Const | 2^22 | 88.975 us | 0.76% | 66.131 us | 1.47% | -22.845 us | -25.68% | FAIL |
| U64 | 2048 | Const | 2^24 | 254.391 us | 0.36% | 167.721 us | 0.64% | -86.670 us | -34.07% | FAIL |
| U64 | 2048 | Const | 2^26 | 932.774 us | 0.11% | 564.983 us | 0.24% | -367.791 us | -39.43% | FAIL |
| U64 | 2048 | Const | 2^28 | 3.649 ms | 0.05% | 2.147 ms | 0.13% | -1502.113 us | -41.16% | FAIL |
| U64 | 2048 | Rand | 2^22 | 88.539 us | 0.72% | 65.744 us | 0.95% | -22.795 us | -25.75% | FAIL |
| U64 | 2048 | Rand | 2^24 | 254.058 us | 0.28% | 168.088 us | 0.48% | -85.970 us | -33.84% | FAIL |
| U64 | 2048 | Rand | 2^26 | 978.114 us | 1.05% | 580.571 us | 0.17% | -397.542 us | -40.64% | FAIL |
| U64 | 2048 | Rand | 2^28 | 4.002 ms | 0.70% | 2.294 ms | 0.15% | -1708.161 us | -42.68% | FAIL |
```
Contributor guide
Assessment
This issue has not been assessed yet.