apache / apache/iceberg-rust

[EPIC] feat: support metadata tables

Open
#823 15 comments 12 reactions 0 assignees View on GitHub
enhancement epic
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

In Iceberg Spark/Flink there are metadata tables that provide information around the table: https://iceberg.apache.org/docs/latest/spark-queries/#inspecting-tables

Supporting this in `iceberg-rust` allows other engines (like RisingWave) to support these "standard" metadata tables.

reference implementation:
-
- https://github.com/apache/iceberg-python/blob/main/pyiceberg/table/inspect.py#L58-L90

List of all metadata tables: https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/MetadataTableType.java#L23
- ENTRIES #863
- FILES
- DATA_FILES
- DELETE_FILES
- HISTORY #841
- METADATA_LOG_ENTRIES #846
- SNAPSHOTS #822
- REFS
- MANIFESTS #861
- PARTITIONS
- ALL_DATA_FILES
- ALL_DELETE_FILES
- ALL_FILES
- ALL_MANIFESTS
- ALL_ENTRIES
- POSITION_DELETES

Contributor guide

Open the contributing guide

Research direction

Start with the Iceberg Spark metadata-table documentation, the PyIceberg inspecting-tables implementation, and MetadataTableType.java. Review the listed subissues, including ENTRIES, HISTORY, SNAPSHOTS, METADATA_LOG_ENTRIES, and MANIFESTS, to determine the current scope. Done means the metadata tables listed in this epic are supported in iceberg-rust.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
40/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.