apache / apache/hudi-rs

Edge case for zero ordering value

Open
#358 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Rust
Stars
279
Forks
67
Avg merge
3d 1h
Merged PRs (30d)
40

Description

Common case of comparing zero ordering value is: delete block contain 0 as the ordering value, and the base record has , say 100, as the ordering. The ordering value will be ignored, commit time will be used for order comparison.

An ordering value of a delete log block could be non-zero, say 100, and the existing ordering (from update data block or the base file) of the record could be 0. Currently, we treat the delete having higher order, thus delete the record.

For aligning on the semantic of treating 0 ordering value, should the 0 value from existing records be also treated as non-comparable, and fall back to commit time ordering?

Contributor guide

Open the contributing guide

Research direction

No file, test, or entry point is named. Start by locating the ordering comparison for delete log blocks and existing records; done means zero ordering values fall back to commit-time comparison consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.