apache / apache/iceberg-rust

Support tables which live on HDFS

Open
#2,440 1 comment 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

### Is your feature request related to a problem or challenge?

Iceberg tables can live on HDFS backends, which is supported by `HadoopFileIO` in Java. Let's build an equivalent here!

### Describe the solution you'd like

Go through OpenDAL using `hdfs-native` and closely follow our existing template for setting up support for this storage system with accompanying tests.

### Willingness to contribute

I can contribute to this feature independently

Contributor guide

Open the contributing guide

Research direction

Locate the existing storage-system integration template and its accompanying tests, then compare the Java HadoopFileIO support with the Rust implementation path through OpenDAL's hdfs-native service. Add equivalent HDFS table support and tests that demonstrate the storage system works; done means the new integration follows the existing template and the tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.