NVIDIA / NVIDIA/cudf

[FEA] Implement Java class `NestedColumnVector` to allow creating nested columns from existing `ColumnVector` objects without data copying

Open
#20,944 0 comments 1 reaction 2 assignees Claimed by @ttnghia View on GitHub
feature request Java Spark
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
Currently, the Java class `ColumnVector` does not support a `release()` method similar to its C++ counterpart. As such, it cannot "release" the native column object to move it to be child of a newly created column. Because of that, we cannot create a nested column from existing `ColumnVector` objects. As such, most of the time, we have to create a new nested columns by copying children from the existing columns.

**Describe the solution you'd like**
By implementing `NestedColumnVector` class derived from `ColumnVector` which just holds a list of references to children columns of type `ColumnVector`, we can effectively have a zero-copy nested column that can be created from existing `ColumnVector` objects:
* Create `NestedColumnVector` Java class derived from `ColumnVector` class, constructed from a data type and a list of `ColumnVector` objects (constructor takes `(DataType type, ColumnVector ...children)` parameter). The data type must be either `STRUCT` or `LIST`. The constructor also calls `incRefCount` on the children columns.
* The class has a private list of order-preserved references to the children columns of type `ColumnVector`.
* The class has a `getNativeViewHandle()` method that create and return pointer to `cudf::column_view` for the nested column it is representing.
* Destructor of this class will also call `close()` on the children columns to decrement their reference counts.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.