[FEA][JNI] Add an API that allows us to build batches on the GPU from host columns
Open
0 - Backlog
feature request
Java
Spark
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
We currently use the ColumnBuilder API in `HostColumnVector.java` to build batches on the spark side. The problem with this API is that as soon as an attempt is made to put host data onto the GPU, the host data is put in a try-with-resources block that closes it. If the creation of the GPU data fails, or a subsequent column in a batch fails to materialize, we cannot retry it (ColumnBuilder does not allow me to build an idempotent BatchBuilder).
This task is to likely add a BatchBuilder that allows us to control when/if the host data is freed, so we can work in retry semantics like https://github.com/NVIDIA/spark-rapids/issues/7851.
Contributor guide
Assessment
This issue has not been assessed yet.