Complete deletion vector feature support
- 主要言語
- C++
- スター
- 221
- フォーク
- 124
- 平均マージ
- 1日 16時間
- マージ済み PR(30日)
- 21
説明
## Goal
Track the remaining work required for complete Iceberg v3 deletion-vector support in iceberg-cpp.
This is a dedicated follow-up to #637 and builds on the initial implementation in #777.
## Summary
| ID | Task | PR | Dependencies |
|---|---|---:|---|
| DV-PR-1 | Cross-language Puffin DV fixtures | [#913](https://github.com/apache/iceberg-cpp/pull/913) | — |
| DV-PR-2 | Puffin metadata validation | [#914](https://github.com/apache/iceberg-cpp/pull/914) | DV-PR-1 |
| DV-PR-3 | Scan applicability validation | [#915](https://github.com/apache/iceberg-cpp/pull/915) | DV-PR-1 |
| DV-PR-4 | Table-aware DV write pipeline | [#919](https://github.com/apache/iceberg-cpp/pull/919) | DV-PR-2, DV-PR-3 |
| DV-PR-5 | `position_deletes` metadata table | [#920](https://github.com/apache/iceberg-cpp/pull/920) | DV-PR-2 |
| DV-PR-6 | Full non-negative int64 positions | [#917](https://github.com/apache/iceberg-cpp/pull/917) | DV-PR-1 |
| DV-PR-7 | DV-aware compaction planner | [#918](https://github.com/apache/iceberg-cpp/pull/918) | DV-PR-3 |
| DV-PR-8 | DV-aware compaction executor | [#921](https://github.com/apache/iceberg-cpp/pull/921) | DV-PR-4, DV-PR-7 |
## Roadmap
### DV-PR-1 — Cross-language Puffin deletion-vector fixtures
PR: [#913](https://github.com/apache/iceberg-cpp/pull/913)
Add independently generated Go and Java Puffin fixtures covering single-blob and multi-blob deletion vectors. Verify Puffin metadata, manifest ranges, referenced data files, cardinalities, high-32-bit keys, Roaring container types, and decoded positions.
### DV-PR-2 — Validate deletion vectors against Puffin metadata
PR: [#914](https://github.com/apache/iceberg-cpp/pull/914)
Dependencies: DV-PR-1
Validate that manifest offsets and sizes identify the expected `deletion-vector-v1` Puffin blob. Check snapshot and sequence IDs, compression, referenced data file, cardinality, blob boundaries, and trailing payload data.
### DV-PR-3 — Enforce partition/spec-aware scan applicability
PR: [#915](https://github.com/apache/iceberg-cpp/pull/915)
Dependencies: DV-PR-1
Match deletion vectors to data files using the complete v3 applicability rules: referenced path, sequence number, partition spec ID, and partition tuple. Ensure applicable DVs supersede older file-scoped position deletes without applying mismatched vectors.
### DV-PR-4 — Add a table-aware v3 deletion-vector write pipeline
PR: [#919](https://github.com/apache/iceberg-cpp/pull/919)
Dependencies: DV-PR-2, DV-PR-3
Provide a table-aware position-delete update API that resolves the target data-file spec and partition, writes v3 deletion vectors, preserves the v2 Parquet path, merges existing file-scoped deletes, removes superseded entries through `RowDelta`, validates conflicts, and performs retry-safe output cleanup.
### DV-PR-5 — Add a `position_deletes` metadata table
PR: [#920](https://github.com/apache/iceberg-cpp/pull/920)
Dependencies: DV-PR-2
Expose logical position-delete rows through a general metadata table regardless of whether they are stored in Parquet files or Puffin deletion vectors. Cover partition evolution, optional deleted-row payloads, mixed upgraded tables, empty scans, and top-level projection.
### DV-PR-6 — Support all legal non-negative int64 positions
PR: [#917](https://github.com/apache/iceberg-cpp/pull/917)
Dependencies: DV-PR-1
Replace dense high-32-bit bitmap storage with sparse ordered buckets so every position from zero through `INT64_MAX` is representable without storage proportional to the largest key. Preserve wire compatibility and serialize only populated buckets in key order.
### DV-PR-7 — Add a DV-aware compaction planner
PR: [#918](https://github.com/apache/iceberg-cpp/pull/918)
Dependencies: DV-PR-3
Plan deterministic, snapshot-bound compaction groups within partition/spec boundaries. Select files using size and file-scoped delete pressure, account for DV cardinality and position deletes, exclude equality deletes, validate configuration, and shape groups around the target output size.
### DV-PR-8 — Add DV-aware compaction execution
PR: [#921](https://github.com/apache/iceberg-cpp/pull/921)
Dependencies: DV-PR-4, DV-PR-7
Execute planned groups by reading source files with all applicable deletes, writing compacted data, and atomically replacing source files while removing obsolete delete entries. Preserve row-lineage semantics, reject stale plans, protect shared Puffin objects, and clean up outputs safely across failures and unknown commit states.
## Delivery
Each item uses an independent worktree, branch, and pull request. Work is performed blockers-first, with focused tests in every PR.
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
調査の方向性
ロードマップと依存関係のリンクから始め、DV-PR-1 から DV-PR-8 を 1 つのタスクではなく個別の作業項目として扱います。issue にはファイルやテストパスが記載されていないため、まず関連するリンク先の PR とその依存関係を確認してください。完了とは、選択した作業項目のスコープに含まれる動作が、独立した worktree、branch、pull request で、焦点を絞ったテストとともに実装されていることを意味します。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- cpp
- 領域
- databases
- issue の種類
- 機能追加
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 20/100