apache / apache/hudi

NullPointerException in shouldKeepNewerRecord when the ordering field value is null

Open
#19,636 0 comments 0 reactions 1 assignee Claimed by @lokeshj1703 View on GitHub
area:reader priority:high
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

## Problem

When a table's ordering (precombine) field is a nullable column, a record whose ordering value is null triggers a `NullPointerException` in the file group reader's merge path.

`OrderingValues.create` maps the field value through the key-generation / field-mapping function and, for the single-field case, returns whatever the function produced. When the underlying column value is null the function returns null, so `create` returns null and the resulting `BufferedRecord` carries a null ordering value.

That null then reaches `BufferedRecordMergerFactory.shouldKeepNewerRecord`, which calls `getOrderingValue().compareTo(...)` without a null guard. During a partition-path update index merge (`HoodieIndexUtils.mergeForPartitionUpdatesAndDeletionsIfNeeded` -> `EventTimeRecordMerger.finalMerge` -> `shouldKeepNewerRecord`) this throws:

```
java.lang.NullPointerException
at org.apache.hudi.common.table.read.BufferedRecordMergerFactory.shouldKeepNewerRecord(...)
```

A nullable ordering column is common in practice (for example a nullable BIGINT event-time column), so any MOR table with such a column can hit this on reads/merges.

## Secondary issue

Coercing the null ordering value to the default (`HoodieRecord.DEFAULT_ORDERING_VALUE`, an `Integer 0`) is not sufficient on its own: `shouldKeepNewerRecord` and `EventTimePartialRecordMerger.finalMerge` then compare that `Integer 0` against a real `Long` ordering value and throw `ClassCastException: java.lang.Long cannot be cast to java.lang.Integer`. Both comparison sites need an `isSameClass` guard (as `deltaMergeDeleteRecord` already has) so a default/type-mismatched ordering value is treated as a tie rather than compared directly.

## Expected behavior

A null ordering field value should not crash reads or merges. It should be treated as the default ordering value, and ordering comparisons should be skipped (record treated as a tie, newer kept) when either value is the default or the two values are of different types.

## Environment

- Affects the 1.x line (file group reader / `BufferedRecordMergerFactory`).
- Merge mode: `EVENT_TIME_ORDERING` (and the partial-update variant when `hoodie.partial.update.mode` is set).

## Resolution

Fixed in #19637. `OrderingValues.isBaseOrderingHigher` centralizes the event-time comparison so a null or default ordering value defers to natural order (the incoming record wins) rather than being coerced to the `int` default and compared against a real value of another type. Coercing the null to the default sentinel at construction (the earlier approach) was not sufficient on its own, because the default `int` then collided with a real `Long` in `compareTo`. A null value in a required (event-time) ordering field remains rejected at write time by `HoodieCreateRecordUtils`.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.