NVIDIA / NVIDIA/cudf

[FEA] `contiguous_split` should accept span of split points, rather than vector

Open
#19,293 1 comment 0 reactions 0 assignees View on GitHub
feature request libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**

Currently, `cudf::contiguous_split` requires that the user provide a `std::vector` of split points. Internally, this vector is used to initialise the `contiguous_split_state` struct and then no longer used.

A common usage of `contiguous_split` is to first hash-partition a table (with `cudf::hash_partition`) and then split and pack into serialised data ready for transfer over the network (with `cudf::contiguous_split`).

`cudf::hash_partition` returns a vector of split points, however, these are not the split points that `contiguous_split` wants. Specifically, `hash_partition` returns the "row offsets to the start of each partition in the output table" (i.e. includes a split point at zero), whereas `contiguous_split` wants the split points corresponding to the "internal" split points of the table (i.e. doesn't include a split point at zero).

Consequently to hash partition and then pack a table into pieces, we must copy the vector of splits to slice away the first element.

**Describe the solution you'd like**

This copy could be avoided if we used a `std::span` for the split points in `contiguous_split`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.