NVIDIA / NVIDIA/cudf

[FEA] Batched construction and batched initialization for `cudf::column`

Open
#20,553 3 comments 1 reaction 0 assignees View on GitHub
feature request libcudf Performance
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

In some places (for example, in groupby aggregation), sometimes we need to create a large number of columns for the output. The number of the being created columns can be over hundreds. Calling to the column factory for each column is inefficient in such situations. For example, here is a screenshot of profiling from such situations:

Image


The screenshot above shows NVTX ranges of the column construction and initialization (to zero value) of around 120 columns. The whole process is very trivial but takes a lot of time.

We need a batched constructor for the column factory that can create a large number of columns at once. In addition, we also need a batched initializer for the columns, initializing the newly constructed columns using the given desired values. Both batched constructor and batched initializer need to operate with unified kernels to reduce overhead as much as possible.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.