NVIDIA / NVIDIA/cudf

[FEA] Try using `__grid_constant__` in libcudf.

Open
#13,360 0 comments 5 reactions 0 assignees View on GitHub
feature request libcudf Performance
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**

Some features in libcudf like the AST and nested comparators have pretty hefty kernels. The `__grid_constant__` annotation (available in CUDA 11.7 and higher) may help reduce local memory usage in these features. We build with CUDA >=11.8 so this would be supported in our builds.

https://docs.nvidia.com/cuda/cuda-c-programming-guide/#grid-constant

In short:
> If the address of a _global_ function parameter is taken, the compiler will ordinarily make a copy of the kernel parameter in thread local memory and use the address of the copy, to partially support C++ semantics, which allow each thread to modify its own local copy of function parameters. Annotating a _global_ function parameter with __grid_constant__ ensures that the compiler will not create a copy of the kernel parameter in thread local memory, but will instead use the generic address of the parameter itself. Avoiding the local copy may result in improved performance.

@jrhemstad thinks a fair amount of the register/stack usage in AST and comparator code comes from thread local copies of those structs. @bdice agrees this is worth investigating.

(Some issue text contributed by @jrhemstad.)

**Describe the solution you'd like**
Try adding `__grid_constant__` to AST kernels and nested comparators to see how the kernels' usage of local memory, registers, etc. changes.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.