apache / apache/iceberg-go

RowDelta: Commit skips Deletion Vector validation when nothing is removed

Open
#1,868 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
463
Forks
232
Avg merge
2d 11h
Merged PRs (30d)
121

Description

### Apache Iceberg version

main (development)

### Please describe the bug 🐞

Iceberg's `V3` format allows at most one live `DV` per data file.

Thus, a writer replacing a data file's `DV` must add the new `DV` and remove the superseeded one in the same snapshot.

The `Commit` function in `RowDelta` enforces this rule, but only when the caller also called `RemovedDeletes`. If a caller adds a replacement DV through `AddDeletes` and forgets `RemoveDeletes`, nothing compares the new `DV` against the `DVs` already in the current snapshot, and the commit **succeeds**.

The result is a table with two live `DVs` on one data file. The commit itself looks clean; the damage appears on the next read, where scan planning rejects the table. `can't index multiple deletion vectors for `. The table stays unreadable until the new metadata is repaired by hand.

The same gap accepts a delta that adds two DVs for one data file, producing the identical broken state. Currently, `Transaction.ReplaceFiles` rejects this exact case, so the two write paths currently disagree.

### Status

Will be working on a PR with the fix.

Contributor guide

Open the contributing guide

Research direction

Start by reading the RowDelta Commit path, focusing on how AddDeletes and RemovedDeletes validate deletion vectors already in the current snapshot. Compare that behavior with Transaction.ReplaceFiles, which the issue identifies as rejecting duplicate DVs. Done means invalid replacement or multiple live DVs are rejected consistently instead of creating an unreadable table.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.