Implement Multiple Prefix Chaining for RewriteTablePath Action
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Feature Request / Improvement
### Overview
Enhanced Apache Iceberg's RewriteTablePathSparkAction to support multiple source-target prefix pairs through a fluent chaining API, enabling complex table migration scenarios with hierarchical path mappings.
### Problem Statement
The original implementation only supported a single source-target prefix pair, limiting users to simple one-to-one path transformations. This was insufficient for:
- Multi-cloud migrations with different storage systems
- Complex data reorganization with multiple path hierarchies
- Cross-environment moves requiring multiple prefix mapping
- Multiple hadoop clusters support
### Usage
// Before: Single prefix only
.rewriteLocationPrefix(sourcePrefix, targetPrefix)
// After: Chainable multiple prefixes
.rewriteLocationPrefix("s3://old-bucket/", "s3://new-bucket/")
.rewriteLocationPrefix("hdfs://cluster/", "s3://data-lake/")
.rewriteLocationPrefix("/tmp/", "s3://staging/")
### Query engine
Spark
### Willingness to contribute
- [x] I can contribute this improvement/feature independently
- [x] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
Contributor guide
Research direction
Start with the RewriteTablePathSparkAction entry point and trace how rewriteLocationPrefix currently stores and applies its source-target mapping. Define completion around supporting chained mappings for the migration examples while preserving the existing single-prefix usage; the issue does not name a test file, so locate the action's existing tests before making changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100