apache / apache/iceberg-rust

Scan projects a stale schema after a schema update with no write

Open
#2,905 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

### Apache Iceberg Rust version

None

### Describe the bug

Schema updates don't create snapshots. So after ADD COLUMN with no write since, the current snapshot still references the older schema — and scanning the current state used it:

// table (id, name) with a row written, then a column added
table.scan().select(["email"]).build() // Err: Column email not found in table
table.scan().select_all().build() // silently returns only (id, name)

email is a real column on the table, but invisible to the scan. Any subsequent write hides the problem, which is why it went unnoticed.

### To Reproduce

for an existing table, add a column, then try to read from that column (error)

### Expected behavior

Should return column with null

### Willingness to contribute

I can contribute a fix for this bug independently

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with an existing table: write a row, add a column without writing again, then run table.scan().select(["email"]).build() and table.scan().select_all().build(). Trace how these scan entry points choose the schema; done means the new column is visible and returns null when it has no values.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.