apache / apache/iceberg-rust

Add support for null values in `Datum`

Open
#1,914 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

In #1824 we use instances of `Datum` to represent constants which can be added either by partition data or by user request as columns in the record batch transformer. However, `Datum` does not support null values. See [this](https://github.com/apache/iceberg-rust/pull/1824#discussion_r2584497197) discussion thread.

Contributor guide

Open the contributing guide

Research direction

Start by reading the Datum implementation and the record batch transformer context described in issue #1824, then review the linked discussion about null values. Define the required null representation and verify that constants from partition data or user requests can be handled as null values in transformed record batches.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.