RowDelta: Commit skips Deletion Vector validation when nothing is removed
- Dominant language
- Go
- Stars
- 463
- Forks
- 232
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 121
Description
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
Iceberg's `V3` format allows at most one live `DV` per data file.
Thus, a writer replacing a data file's `DV` must add the new `DV` and remove the superseeded one in the same snapshot.
The `Commit` function in `RowDelta` enforces this rule, but only when the caller also called `RemovedDeletes`. If a caller adds a replacement DV through `AddDeletes` and forgets `RemoveDeletes`, nothing compares the new `DV` against the `DVs` already in the current snapshot, and the commit **succeeds**.
The result is a table with two live `DVs` on one data file. The commit itself looks clean; the damage appears on the next read, where scan planning rejects the table. `can't index multiple deletion vectors for `. The table stays unreadable until the new metadata is repaired by hand.
The same gap accepts a delta that adds two DVs for one data file, producing the identical broken state. Currently, `Transaction.ReplaceFiles` rejects this exact case, so the two write paths currently disagree.
### Status
Will be working on a PR with the fix.
Contributor guide
Research direction
Start by reading the RowDelta Commit path, focusing on how AddDeletes and RemovedDeletes validate deletion vectors already in the current snapshot. Compare that behavior with Transaction.ReplaceFiles, which the issue identifies as rejecting duplicate DVs. Done means invalid replacement or multiple live DVs are rejected consistently instead of creating an unreadable table.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100