apache / apache/iceberg-rust

Update a TableSchema from a Schema

Open
#698 4 comments 1 reaction 0 assignees View on GitHub
not-stale
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

To make things easier for query engines to evolve schemas into a target schema, it would be great to add `UnionByName` to Iceberg-Rust as well. The `UnionByName` visitor ([python](https://github.com/apache/iceberg-python/blob/2ba86b5c852cb0c8f4b4ba95d54fb5a6bfa00fa2/pyiceberg/table/update/schema.py#L722), [java](https://github.com/apache/iceberg/blob/daa24f9c3a56e18d188097deb2dd79cc991c9a78/core/src/main/java/org/apache/iceberg/schema/UnionByNameVisitor.java#L33)) to make extending of the schema easy. For example, see the [Python docs on Union by Name](https://py.iceberg.apache.org/api/#union-by-name).

For example, if you have a CDC application, and you see a new schema, you can just pass it into this method to update the table. It will traverse the schema and compare it based on the names.

Depends on #697

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked Python and Java UnionByName implementations, the Python Union by Name documentation, and dependency issue #697. Define completion as adding an Iceberg-Rust operation that traverses schemas by field names and updates a TableSchema toward a target schema, including the CDC use case described here.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.