Update a TableSchema from a Schema
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
To make things easier for query engines to evolve schemas into a target schema, it would be great to add `UnionByName` to Iceberg-Rust as well. The `UnionByName` visitor ([python](https://github.com/apache/iceberg-python/blob/2ba86b5c852cb0c8f4b4ba95d54fb5a6bfa00fa2/pyiceberg/table/update/schema.py#L722), [java](https://github.com/apache/iceberg/blob/daa24f9c3a56e18d188097deb2dd79cc991c9a78/core/src/main/java/org/apache/iceberg/schema/UnionByNameVisitor.java#L33)) to make extending of the schema easy. For example, see the [Python docs on Union by Name](https://py.iceberg.apache.org/api/#union-by-name).
For example, if you have a CDC application, and you see a new schema, you can just pass it into this method to update the table. It will traverse the schema and compare it based on the names.
Depends on #697
Contributor guide
Research direction
Start by reading the linked Python and Java UnionByName implementations, the Python Union by Name documentation, and dependency issue #697. Define completion as adding an Iceberg-Rust operation that traverses schemas by field names and updates a TableSchema toward a target schema, including the CDC use case described here.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100