apache / apache/iceberg-cpp

Support column default values (initial-default / write-default)

Open
#730 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
221
Forks
124
Avg merge
1d 16h
Merged PRs (30d)
21

Description

Sub-issue of #637 (item 2: Column Default Values).
Iceberg v3 allows schema evolution with column default values (spec — Default values (https://iceberg.apache.org/spec/#default-values)):
- initial-default: returned for rows that predate the column's addition, i.e. applied on the read path when a data file does not contain the field.
- write-default: applied on the write path when a writer does not supply a value or the column.

Planned scope:
- [x] Schema model: carry initial-default / write-default on SchemaField
- [x] JSON serde: parse/write the two fields using single-value serialization
- [x] Read path: project missing columns as the initial-default constant instead of null when reading data files written before the column existed
- [x] Write path: apply write-default where values are omitted; validation (e.g. adding a required column requires an initial-default, v3+ gating)
I plan to work on this and will send PR(s) referencing this issue.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Iceberg v3 default-values specification and trace SchemaField, JSON serde, and the read and write paths named in the scope. Done means initial-default and write-default behavior is implemented, including missing-column reads, omitted-value writes, and the stated required-column and version validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.