NVIDIA / NVIDIA/cccl

[FEA]: Tune `DeviceMemcpy` on H100

Open
#237 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

### Is this a duplicate?

- [X] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cccl/issues) for this request and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)

### Area

CUB

### Is your feature request related to a problem? Please describe.

`DeviceMemcpy` is currently not tuned for H100. Preliminary research shows that there's some tuning potential:

```
variant score mins means maxs
tpb_128.bpt_1.tlevbpt_2.ltpb_160.lbbpt_76.ppb_0.wlt_96.blt_8192.blns_1416.bldcid_5.bll2w_965.buns_1708.budcid_6.bul2w_205 (1.db) 1.113028 0.632304 1.109764 2.732963
```
![image](https://github.com/NVIDIA/cccl/assets/9890394/c20c9c9b-dd45-4490-84d8-edaaf50d107e)

```
variant score mins means maxs
tpb_256.bpt_2.tlevbpt_6.ltpb_160.lbbpt_112.ppb_0.wlt_192.blt_10752.blns_1596.bldcid_6.bll2w_945.buns_488.budcid_1.bul2w_330 (5.db) 1.084122 0.873292 1.059192 1.682492
```

![image](https://github.com/NVIDIA/cccl/assets/9890394/3f0b9e84-96fc-4156-af23-090352e4585d)

```
variant score mins means maxs
tpb_128.bpt_1.tlevbpt_2.ltpb_800.lbbpt_100.ppb_0.wlt_32.blt_5120.blns_1888.bldcid_3.bll2w_510.buns_1852.budcid_6.bul2w_1000 (5.db) 1.123700 0.714974 1.111115 3.054015
```

![image](https://github.com/NVIDIA/cccl/assets/9890394/60bd0349-d602-461a-b7e3-5f90d54cb79c)

The common scheme behind these results is better performance on large segments (up to 3x), and up to 30% regression for small segments. We need more resources to run a more extended search.

### Describe the solution you'd like

We need an SM90 tuning policy in `DeviceMemcpy`.

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.