apache / apache/iceberg-python

Support rewrite_manifests table maintenance

Open
#3,629 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.1k
Forks
581
Avg merge
1d 17h
Merged PRs (30d)
78

Description

### Feature Request / Improvement

Rewriting manifests without changing data speeds up scan planning when a table has accumulated many small manifests. Java exposes this as `Table.rewriteManifests()` (Spark: `rewrite_manifests`).

Scope: an explicit maintenance operation that reads the current snapshot's manifests, regroups entries, rewrites them preserving sequence numbers (entries rewritten as EXISTING) and, for V3, `first_row_id`, and commits a `replace` snapshot. Related: #270 tracks automatic manifest merging on write; this covers the explicit maintenance API.

Contributor guide

No contributing guide indexed for this repository

Research direction

Locate the Python table maintenance/API entry point corresponding to Table.rewriteManifests(); read the current snapshot and manifest-management paths first. Done means an explicit operation regroups current manifests, preserves sequence numbers and V3 first_row_id, marks rewritten entries EXISTING, and commits a replace snapshot.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.