apache / apache/iceberg-cpp

feat: implement ReplacePartitions (two-PR plan)

Open
#775 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
221
Forks
124
Avg merge
1d 16h
Merged PRs (30d)
21

Description

## Overview

This issue tracks the implementation of `ReplacePartitions` in C++, assigned in #637.

`ReplacePartitions` performs dynamic partition overwrite: each file added to the operation marks its entire partition for deletion, and on commit, existing files in those partitions are atomically replaced. The resulting snapshot uses `operation="overwrite"` and summary property `"replace-partitions"="true"`.

## Implementation Plan

### PR 1 — Core implementation

New files:
- `src/iceberg/update/replace_partitions.h`
- `src/iceberg/update/replace_partitions.cc`

Modified files:
- `src/iceberg/snapshot.h` — add `kReplacePartitions` constant to `SnapshotSummaryFields`
- `src/iceberg/type_fwd.h` — add `class ReplacePartitions;`
- `src/iceberg/CMakeLists.txt` — register new source

### PR 2 — Table/Transaction API surface + tests

Modified files:
- `src/iceberg/table.h` / `table.cc` — add `NewReplacePartitions()`
- `src/iceberg/transaction.h` / `transaction.cc` — add `NewReplacePartitions()`

New files:
- `src/iceberg/test/replace_partitions_test.cc`
- `src/iceberg/test/CMakeLists.txt` — register test

## Design

`ReplacePartitions` extends `SnapshotUpdate` (same pattern as `FastAppend`) and uses `ManifestFilterManager` to mark existing partition files as DELETED:

- `AddFile(file)`: calls `filter_manager_.DropPartition(spec_id, partition)`, accumulates new file in `new_data_files_by_spec_`
- `ValidateAppendOnly()`: calls `filter_manager_.FailAnyDelete()`
- `Apply()`: for unpartitioned tables calls `filter_manager_.DeleteByRowFilter(AlwaysTrue)`; calls `FilterManifests()` on existing snapshot, then prepends new manifests
- `operation()`: returns `DataOperation::kOverwrite`
- `Summary()`: sets `"replace-partitions"="true"`

Relates to: #637

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the existing FastAppend and SnapshotUpdate implementations, then inspect the listed snapshot, manifest, table, and transaction files. Use the planned replace_partitions_test.cc and its CMake registration to verify partition replacement, unpartitioned behavior, operation and summary metadata, and the new table and transaction APIs are covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.