apache / apache/iceberg

Implement Multiple Prefix Chaining for RewriteTablePath Action

Open
#14,318 3 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Feature Request / Improvement

### Overview

Enhanced Apache Iceberg's RewriteTablePathSparkAction to support multiple source-target prefix pairs through a fluent chaining API, enabling complex table migration scenarios with hierarchical path mappings.

### Problem Statement

The original implementation only supported a single source-target prefix pair, limiting users to simple one-to-one path transformations. This was insufficient for:
- Multi-cloud migrations with different storage systems
- Complex data reorganization with multiple path hierarchies
- Cross-environment moves requiring multiple prefix mapping
- Multiple hadoop clusters support

### Usage

// Before: Single prefix only
.rewriteLocationPrefix(sourcePrefix, targetPrefix)

// After: Chainable multiple prefixes
.rewriteLocationPrefix("s3://old-bucket/", "s3://new-bucket/")
.rewriteLocationPrefix("hdfs://cluster/", "s3://data-lake/")
.rewriteLocationPrefix("/tmp/", "s3://staging/")

### Query engine

Spark

### Willingness to contribute

- [x] I can contribute this improvement/feature independently
- [x] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time

Contributor guide

Open the contributing guide

Research direction

Start with the RewriteTablePathSparkAction entry point and trace how rewriteLocationPrefix currently stores and applies its source-target mapping. Define completion around supporting chained mappings for the migration examples while preserving the existing single-prefix usage; the issue does not name a test file, so locate the action's existing tests before making changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.