[FEA] Batched construction and batched initialization for `cudf::column`
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
In some places (for example, in groupby aggregation), sometimes we need to create a large number of columns for the output. The number of the being created columns can be over hundreds. Calling to the column factory for each column is inefficient in such situations. For example, here is a screenshot of profiling from such situations:
The screenshot above shows NVTX ranges of the column construction and initialization (to zero value) of around 120 columns. The whole process is very trivial but takes a lot of time.
We need a batched constructor for the column factory that can create a large number of columns at once. In addition, we also need a batched initializer for the columns, initializing the newly constructed columns using the given desired values. Both batched constructor and batched initializer need to operate with unified kernels to reduce overhead as much as possible.
Contributor guide
Assessment
This issue has not been assessed yet.