[VL] Story: Improve on data ingestion workloads
Open
DATA_LAKE
enhancement
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Description
Currently Gluten supports reading on Iceberg/Hudi/Delta tables natively. However on the write path Gluten does not support these and will use JVM Spark code.
On Iceberg side, Velox community already have a working patch with write support(non-bucketed): https://github.com/facebookincubator/velox/pull/10996
Note this heavily depends on the C++ write support of Iceberg/Hudi/Delta. A unified design will beneficial for all backends.
Contributor guide
Assessment
This issue has not been assessed yet.