[FEA] Java ColumnVector should support async copyToHost
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
During Spark's shuffle table columns are copied back to the host via `copyToHost`. Each operation is synchronous with the default stream. This causes unnecessary synchronization when there are many columns to copy, since only a single synchronization at the end of all copy transfers is necessary. Also the copy interface does not allow a stream to be specified.
**Describe the solution you'd like**
`ColumnVector` should provide a `copyToHostAsync` method that can specify an optional stream to use for the copy operations. The Javadoc for the method should make it clear that the caller must synchronize with the stream before using the resulting `HostColumnVector` instance.
**Describe alternatives you've considered**
This can be accomplished today via reaching into the `ColumnVector` and accessing the individual `DeviceMemoryBuffer` instances (e.g.: data, validity, offsets), but this is complicated for the caller and especially so when child columns are involved for nested types.
Contributor guide
Assessment
This issue has not been assessed yet.