apache / apache/gravitino

[FEATURE] Add capability to fetch partitions from Iceberg tables

Open
#3,840 4 comments 2 reactions 0 assignees View on GitHub
feature
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 15h
Merged PRs (30d)
315

Description

### Describe the feature

Add capability to fetch partitions from Iceberg tables to enable easy to use and efficient mechanism to fetch Iceberg table partitions.

### Motivation

It is common for users to build use-cases, like waiters, that depend on partitions information of a table. While users are moving from hive to iceberg table format, one of the blockers they see is the ease and speed of partitions access information. Very large iceberg tables (with multiple 10Ks) partitions takes over an hour and over 50g of memory.

### Describe the solution

Add rest endpoint to get Iceberg partitions.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by locating the existing REST endpoints and Iceberg table integration, then determine the partition response contract and an efficient retrieval approach; done means a REST endpoint exposes Iceberg table partitions without the reported large-table performance cost.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.