apache / apache/gluten

[VL] Story: Improve on data ingestion workloads

Open
#9,225 1 comment 0 reactions 0 assignees View on GitHub
DATA_LAKE enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Description

Currently Gluten supports reading on Iceberg/Hudi/Delta tables natively. However on the write path Gluten does not support these and will use JVM Spark code.

On Iceberg side, Velox community already have a working patch with write support(non-bucketed): https://github.com/facebookincubator/velox/pull/10996

Note this heavily depends on the C++ write support of Iceberg/Hudi/Delta. A unified design will beneficial for all backends.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.