NVIDIA / NVIDIA/cccl

[FEA]: Runtime interface to query shared memory requirement

Open
#5,432 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

### Is this a duplicate?

- [x] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cccl/issues) for this request and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)

### Area

CUB

### Is your feature request related to a problem? Please describe.

When making a decision on whether to JIT-compile CUB code for a device reduction, we only know the size of the reduction at compile time. However, we need to know shared memory usage at runtime to determine if we should take the JIT path or fall back to non-JIT. Currently we have to compile a small program to obtain this info, which is a non-deterministic amount of overhead. It would be useful if there was a runtime library similar to libmathdx where this information can be queried at runtime. It doesn't have to be 100% accurate due to alignment issues, but an overestimate is better than an underestimate.

### Describe the solution you'd like

Runtime function taking ITEMS_PER_THREAD and type as inputs, and outputs shared memory requirements in bytes for each CUB device function.

### Describe alternatives you've considered

Compiling a small program for shm size

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.