apache / apache/gravitino

[Improvement] Add Support for Inspecting Tables in Datalake Formats like Iceberg

Open
#4,798 13 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### What would you like to be improved?

In addition to regular table functions, datalake table formats offer various capabilities for inspecting tables.

For instance, Iceberg can display valid snapshots for a table or show a table's current file manifests.

However, Gravitino catalogs currently lack this support, and there is no designated place for it in the General API hierarchy.

Incorporating this support into Gravitino would help users better manage their datalakes.

### How should we improve?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing Gravitino's General API hierarchy and the existing catalog table functions to determine where datalake inspection belongs. Compare the capabilities described for Iceberg, including valid snapshots and current file manifests. Done means a defined API location and support for inspecting these table details in Gravitino catalogs.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.