NVIDIA / NVIDIA/cudf

[FEA] chunked_pack should allow variable sized spans in the next call

Open
#13,332 1 comment 0 reactions 1 assignee Claimed by @abellina View on GitHub
0 - Backlog feature request libcudf Spark
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

As brought up here https://github.com/rapidsai/cudf/pull/13260#discussion_r1189086902, `chunked_pack` requires that it be configured with a single size for the user's bounce buffer, and further requires the same size of device memory made available in every call to `chunked_pack::next`.

This is overkill in the case where we have actually very little work to be done (the source table is small). So the proposal is to allow for a `get_next_size()` api that lets us know the size of buffer that should be made available to `next`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.