NVIDIA / NVIDIA/cccl

[FEA]: Specialize `Block{Load,Store[,Exchange]}` when `ITEMS_PER_THREAD` is `1`

Open
#1,127 1 comment 2 reactions 0 assignees View on GitHub
cub
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

### Is this a duplicate?

- [X] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cccl/issues) for this request and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)

### Area

CUB

### Is your feature request related to a problem? Please describe.

`BlockLoad` provides a means to load data into a blocked arrangement. The following `BlockLoadAlgorithm` strategies load data with a striped, memory access-friendly pattern (i.e., neighbouring threads access neighbouring items in memory) followed by a `BlockExchange` to get the data from a striped (or warp-striped) arrangement into a blocked arrangement:
- `BLOCK_LOAD_TRANSPOSE`
- `BLOCK_LOAD_WARP_TRANSPOSE`
- `BLOCK_LOAD_WARP_TRANSPOSE_TIMESLICED`

When `ITEMS_PER_THREAD` is `1`, blocked and striped arrangements are equivalent. This means that (1) no data exchange amongst threads is actually needed and (2) we want to avoid allocating shared memory for `TempStorage`, as we don't need
a scratchpad for data exchange. This applies to `BlockLoad`, `BlockStore`, and `BlockExchange`.

It's worth noting that `BlockExchange` provides the `ScatterTo{Blocked,Striped}` member functions for which we will still require `TempStorage` scratchpad for data exchange. Unfortunately, at the time of class instantiation, we don't know if these member functions will be used and, hence, I'm afraid we will need to keep allocating `TempStorage` scratchpad for `BlockExchange`, even if `ITEMS_PER_THREAD` is `1`.

_Thanks to @gevtushenko for [suggesting this](https://github.com/NVIDIA/cccl/issues/1095#issuecomment-1811908648)._

### Describe the solution you'd like

Avoid superfluous data exchange and `TempStorage` allocations when `ITEMS_PER_THREAD` is `1`.

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.