apache / apache/iceberg-cpp

feat: implement ReplacePartitions (two-PR plan)

未关闭
#775 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
C++
星标
221
派生
124
平均合并
1 天 16 小时
30 天内合并 PR
21

描述

## Overview

This issue tracks the implementation of `ReplacePartitions` in C++, assigned in #637.

`ReplacePartitions` performs dynamic partition overwrite: each file added to the operation marks its entire partition for deletion, and on commit, existing files in those partitions are atomically replaced. The resulting snapshot uses `operation="overwrite"` and summary property `"replace-partitions"="true"`.

## Implementation Plan

### PR 1 — Core implementation

New files:
- `src/iceberg/update/replace_partitions.h`
- `src/iceberg/update/replace_partitions.cc`

Modified files:
- `src/iceberg/snapshot.h` — add `kReplacePartitions` constant to `SnapshotSummaryFields`
- `src/iceberg/type_fwd.h` — add `class ReplacePartitions;`
- `src/iceberg/CMakeLists.txt` — register new source

### PR 2 — Table/Transaction API surface + tests

Modified files:
- `src/iceberg/table.h` / `table.cc` — add `NewReplacePartitions()`
- `src/iceberg/transaction.h` / `transaction.cc` — add `NewReplacePartitions()`

New files:
- `src/iceberg/test/replace_partitions_test.cc`
- `src/iceberg/test/CMakeLists.txt` — register test

## Design

`ReplacePartitions` extends `SnapshotUpdate` (same pattern as `FastAppend`) and uses `ManifestFilterManager` to mark existing partition files as DELETED:

- `AddFile(file)`: calls `filter_manager_.DropPartition(spec_id, partition)`, accumulates new file in `new_data_files_by_spec_`
- `ValidateAppendOnly()`: calls `filter_manager_.FailAnyDelete()`
- `Apply()`: for unpartitioned tables calls `filter_manager_.DeleteByRowFilter(AlwaysTrue)`; calls `FilterManifests()` on existing snapshot, then prepends new manifests
- `operation()`: returns `DataOperation::kOverwrite`
- `Summary()`: sets `"replace-partitions"="true"`

Relates to: #637

贡献指南

这个仓库没有索引到贡献指南

调研方向

首先阅读现有的 FastAppend 和 SnapshotUpdate 实现,然后检查列出的 snapshot、manifest、table 和 transaction 文件。使用计划中的 replace_partitions_test.cc 及其在 CMake 中的注册,验证分区替换、非分区行为、operation 和 summary 元数据,以及新的 table 和 transaction API 都已覆盖。

由索引模型根据 Issue 内容生成。

评估

技术栈
cpp
领域
data-engineering
Issue 类型
功能
难度
4/5
预计耗时
3-5 天
活跃度
冷清
描述清晰度
描述清楚
新手友好度
58/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。