NullPointerException in shouldKeepNewerRecord when the ordering field value is null
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
## Problem
When a table's ordering (precombine) field is a nullable column, a record whose ordering value is null triggers a `NullPointerException` in the file group reader's merge path.
`OrderingValues.create` maps the field value through the key-generation / field-mapping function and, for the single-field case, returns whatever the function produced. When the underlying column value is null the function returns null, so `create` returns null and the resulting `BufferedRecord` carries a null ordering value.
That null then reaches `BufferedRecordMergerFactory.shouldKeepNewerRecord`, which calls `getOrderingValue().compareTo(...)` without a null guard. During a partition-path update index merge (`HoodieIndexUtils.mergeForPartitionUpdatesAndDeletionsIfNeeded` -> `EventTimeRecordMerger.finalMerge` -> `shouldKeepNewerRecord`) this throws:
```
java.lang.NullPointerException
at org.apache.hudi.common.table.read.BufferedRecordMergerFactory.shouldKeepNewerRecord(...)
```
A nullable ordering column is common in practice (for example a nullable BIGINT event-time column), so any MOR table with such a column can hit this on reads/merges.
## Secondary issue
Coercing the null ordering value to the default (`HoodieRecord.DEFAULT_ORDERING_VALUE`, an `Integer 0`) is not sufficient on its own: `shouldKeepNewerRecord` and `EventTimePartialRecordMerger.finalMerge` then compare that `Integer 0` against a real `Long` ordering value and throw `ClassCastException: java.lang.Long cannot be cast to java.lang.Integer`. Both comparison sites need an `isSameClass` guard (as `deltaMergeDeleteRecord` already has) so a default/type-mismatched ordering value is treated as a tie rather than compared directly.
## Expected behavior
A null ordering field value should not crash reads or merges. It should be treated as the default ordering value, and ordering comparisons should be skipped (record treated as a tie, newer kept) when either value is the default or the two values are of different types.
## Environment
- Affects the 1.x line (file group reader / `BufferedRecordMergerFactory`).
- Merge mode: `EVENT_TIME_ORDERING` (and the partial-update variant when `hoodie.partial.update.mode` is set).
## Resolution
Fixed in #19637. `OrderingValues.isBaseOrderingHigher` centralizes the event-time comparison so a null or default ordering value defers to natural order (the incoming record wins) rather than being coerced to the `int` default and compared against a real value of another type. Coercing the null to the default sentinel at construction (the earlier approach) was not sufficient on its own, because the default `int` then collided with a real `Long` in `compareTo`. A null value in a required (event-time) ordering field remains rejected at write time by `HoodieCreateRecordUtils`.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.