NVIDIA / NVIDIA/cudf

[FEA] support host memory in pack/unpack

Open
#20,003 7 comments 2 reactions 0 assignees View on GitHub
feature request libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
when we want to prepare data for external data transfer, we need to pack the data to get it into a contiguous memory segment. unfortunately, this is only possible today to device memory, which means that to pack X GB of data, we need to consume an other X GB of device memory (roughly).
similarly, unpack() today only works on device memory only.

**Describe the solution you'd like**
it would be great if we could get the packed data directly to host memory instead of device memory; and for unpack(), to unpack from host memory to device memory.

**Describe alternatives you've considered**
today we could pack on the device, then copy the memory to host memory, however, this means, we have a memory spike on the device until we can clean it up after the copy. similarly for unpack.
I looked at the python code and the wrapper there does exactly this.

**Additional context**
this came up as an enhancement for our implementation of a UCX based data exchange in Velox. given that we can't always control how much memory we want to send over to a peer, it could be, that pack fails in memory allocation, or it starves other data processing because of the memory spike.

I overall think this would be a great enhancement.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.