[EPIC] feat: support metadata tables
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
In Iceberg Spark/Flink there are metadata tables that provide information around the table: https://iceberg.apache.org/docs/latest/spark-queries/#inspecting-tables
Supporting this in `iceberg-rust` allows other engines (like RisingWave) to support these "standard" metadata tables.
reference implementation:
-
- https://github.com/apache/iceberg-python/blob/main/pyiceberg/table/inspect.py#L58-L90
List of all metadata tables: https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/MetadataTableType.java#L23
- ENTRIES #863
- FILES
- DATA_FILES
- DELETE_FILES
- HISTORY #841
- METADATA_LOG_ENTRIES #846
- SNAPSHOTS #822
- REFS
- MANIFESTS #861
- PARTITIONS
- ALL_DATA_FILES
- ALL_DELETE_FILES
- ALL_FILES
- ALL_MANIFESTS
- ALL_ENTRIES
- POSITION_DELETES
Contributor guide
Research direction
Start with the Iceberg Spark metadata-table documentation, the PyIceberg inspecting-tables implementation, and MetadataTableType.java. Review the listed subissues, including ENTRIES, HISTORY, SNAPSHOTS, METADATA_LOG_ENTRIES, and MANIFESTS, to determine the current scope. Done means the metadata tables listed in this epic are supported in iceberg-rust.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 40/100