[FEA] support host memory in pack/unpack
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
when we want to prepare data for external data transfer, we need to pack the data to get it into a contiguous memory segment. unfortunately, this is only possible today to device memory, which means that to pack X GB of data, we need to consume an other X GB of device memory (roughly).
similarly, unpack() today only works on device memory only.
**Describe the solution you'd like**
it would be great if we could get the packed data directly to host memory instead of device memory; and for unpack(), to unpack from host memory to device memory.
**Describe alternatives you've considered**
today we could pack on the device, then copy the memory to host memory, however, this means, we have a memory spike on the device until we can clean it up after the copy. similarly for unpack.
I looked at the python code and the wrapper there does exactly this.
**Additional context**
this came up as an enhancement for our implementation of a UCX based data exchange in Velox. given that we can't always control how much memory we want to send over to a peer, it could be, that pack fails in memory allocation, or it starves other data processing because of the memory spike.
I overall think this would be a great enhancement.
Contributor guide
Assessment
This issue has not been assessed yet.