feat: implement ReplacePartitions (two-PR plan)
- 主要语言
- C++
- 星标
- 221
- 派生
- 124
- 平均合并
- 1 天 16 小时
- 30 天内合并 PR
- 21
描述
## Overview
This issue tracks the implementation of `ReplacePartitions` in C++, assigned in #637.
`ReplacePartitions` performs dynamic partition overwrite: each file added to the operation marks its entire partition for deletion, and on commit, existing files in those partitions are atomically replaced. The resulting snapshot uses `operation="overwrite"` and summary property `"replace-partitions"="true"`.
## Implementation Plan
### PR 1 — Core implementation
New files:
- `src/iceberg/update/replace_partitions.h`
- `src/iceberg/update/replace_partitions.cc`
Modified files:
- `src/iceberg/snapshot.h` — add `kReplacePartitions` constant to `SnapshotSummaryFields`
- `src/iceberg/type_fwd.h` — add `class ReplacePartitions;`
- `src/iceberg/CMakeLists.txt` — register new source
### PR 2 — Table/Transaction API surface + tests
Modified files:
- `src/iceberg/table.h` / `table.cc` — add `NewReplacePartitions()`
- `src/iceberg/transaction.h` / `transaction.cc` — add `NewReplacePartitions()`
New files:
- `src/iceberg/test/replace_partitions_test.cc`
- `src/iceberg/test/CMakeLists.txt` — register test
## Design
`ReplacePartitions` extends `SnapshotUpdate` (same pattern as `FastAppend`) and uses `ManifestFilterManager` to mark existing partition files as DELETED:
- `AddFile(file)`: calls `filter_manager_.DropPartition(spec_id, partition)`, accumulates new file in `new_data_files_by_spec_`
- `ValidateAppendOnly()`: calls `filter_manager_.FailAnyDelete()`
- `Apply()`: for unpartitioned tables calls `filter_manager_.DeleteByRowFilter(AlwaysTrue)`; calls `FilterManifests()` on existing snapshot, then prepends new manifests
- `operation()`: returns `DataOperation::kOverwrite`
- `Summary()`: sets `"replace-partitions"="true"`
Relates to: #637
贡献指南
这个仓库没有索引到贡献指南
调研方向
首先阅读现有的 FastAppend 和 SnapshotUpdate 实现,然后检查列出的 snapshot、manifest、table 和 transaction 文件。使用计划中的 replace_partitions_test.cc 及其在 CMake 中的注册,验证分区替换、非分区行为、operation 和 summary 元数据,以及新的 table 和 transaction API 都已覆盖。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- cpp
- 领域
- data-engineering
- Issue 类型
- 功能
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 冷清
- 描述清晰度
- 描述清楚
- 新手友好度
- 58/100