apache / apache/iceberg-cpp

Track Iceberg v3 Feature Support (Roadmap for 0.4.0)

未关闭
#637 21 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
C++
星标
221
派生
124
平均合并
1 天 16 小时
30 天内合并 PR
21

描述

We already have **partial support for Iceberg v3** in the codebase (e.g., Row Lineage and partition statistics). However, a number of key v3 features are still missing.

This issue serves as a **parent tracking issue** for the remaining v3 work we plan to (partially) include in the **0.3.0 release**.

If you are interested in working on any of the items below, please **open a dedicated sub-issue** and link it back to this one.

## V3 related

### 1. Schema-Level Support for New v3 Types

Add support for all newly introduced v3 types at the schema level, including:

- [x] `timestamp_ns`, `timestamptz_ns` @zhjwpku
- [x] `unknown` @manuzhang
- [ ] `variant` @zhjwpku
- [x] `geometry`, `geography` @wgtmac

### 2. Column Default Values (Schema Evolution)

Enable schema evolution with default values:

- [x] Support `initial-default` and `write-default` @huan233usc
- [ ] Apply defaults correctly during read and write paths

### ~~3. Multi-Argument Transforms (Not planned, as they are not supported by the Java implementation)~~

### 4. Binary Deletion Vectors

Add support for deletion vectors (DV):

- [x] Puffin-based DV parsing @zhaoxuan1994

### 5. Table-Level Encryption Metadata (Not planned for 0.4.0) @shangxinli

Support encryption-related metadata defined in v3:

- [ ] Parse and preserve encryption metadata

### 6. Puffin-Based Statistics (Not planned for 0.4.0)

Support table and partition statistics stored in Puffin files:

- [ ] Metadata parsing
- [ ] Optional integration into scan planning

## Table update and maintenance: #364

- [x] OverwriteFiles @lishuxu
- [x] RowDelta @manuzhang
- [x] MergeAppend @zhjwpku
- [x] DeleteFiles @manuzhang
- [x] RewriteFiles @WZhuo
- [ ] ReplacePartitions @shangxinli
- [ ] RewriteManifests @zhjwpku
- [ ] CherryPickOperation @shangxinli

## Miscellaneous

- [ ] Metadata table inspection @All-less @WZhuo
- [x] LRU cache @HuaHuaY
- [ ] Logging interface and spdlog integration @kamcheungting-db

If anything is missing or you have suggestions, feel free to comment or open additional sub-issues.

贡献指南

这个仓库没有索引到贡献指南

调研方向

这是一个父级跟踪 issue,而不是一个自包含的任务,其中没有指定文件、测试或入口点。选择一个尚未勾选的项目,例如 schema 默认值、加密元数据或表更新操作,然后打开一个与此处关联的专用子 issue。完成意味着完成该范围明确的项目,并更新其对应的检查清单条目。

由索引模型根据 Issue 内容生成。

评估

技术栈
cpp
领域
data-engineering
Issue 类型
功能
难度
5/5
预计耗时
一周以上
活跃度
冷清
描述清晰度
需要澄清
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。