NVIDIA / NVIDIA/cudf

[FEA] Improve Performance of Transforms on Small Wide Tables

Open
#19,625 0 comments 0 reactions 0 assignees View on GitHub
feature request libcudf Performance
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**

Multi-threaded transforms on small tables (<100K) need improvement.

Running the concurrent polynomial transforms benchmark (https://github.com/rapidsai/cudf/pull/19199) on 100k rows at polynomial order 8 takes 543 us per thread.
Most of the time is spent setting up the pipeline to run the kernel.

Image

A more in-depth profile shows that the time in `build_transform_kernel` is spent copying the Kernel's preprocessed code, due to implicit copies whilst they were being passed across functions, even for the hot runs.
This implicit copy occurs 2x in CUDF and 1x in JITIFY, both for the cold and hot JIT runs, even though the program data is a globally defined static constant.

Removing the implicit copy in CUDF alone yielded a 66% improvement in the total time from 540us to 160us, shown below:


Image

Removing the preprocessed program data in JITIFY would also make the cold JIT runs faster.

In the second improved profile above, we spend 40% of the time in `bitmask_and`.
using `bitmask_and` negates the advantages of JIT-compilation as it internally uses binary operations on each column pair's nullmasks, which is especially bad for wide input columns.
In addition, `bitmask_and` causes a stream synchronization.

We can solve this by computing the bitmasks in the kernel instead. The kernel would use atomic ops to set the null masks of each row (as done in https://github.com/rapidsai/cudf/pull/19199), which should be free since there wouldn't be contention.
This should theoretically bring us to ~80us for processing 100K input columns.

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

- [x] remove implicit copies of the preprocessed program data https://github.com/rapidsai/cudf/pull/19667
- [ ] examine the impact of computing the nullmasks in the kernel instead of using `bitmask_and`
- [x] examine JITify for performance hotspots, especially around copying the program data

Follows-up https://github.com/rapidsai/cudf/issues/18023

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context, code examples, or references to existing implementations about the feature request here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.