Unexpected `Null` behavior for `ScalarValue::partial_cmp()`
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
For `partial_cmp()` function for `ScalarValue`
https://github.com/apache/datafusion/blob/79f67b8ef2d3016c866ae92d7d2f56d1a7830045/datafusion/common/src/scalar/mod.rs#L568
The current behavior is unexpected:
```rust
// this test can be added to the above file that `partial_cmp()` lives
#[test]
fn scalar_partial_ordering_nulls() {
use ScalarValue::*;
assert_eq!(
Int32(Some(3)).partial_cmp(&Int32(None)),
Some(Ordering::Greater)
);
assert_eq!(
Int32(None).partial_cmp(&Int32(Some(3))),
Some(Ordering::Less)
);
}
```
The comparison between `ScalarValue { type: Int32, val: 3 }` and `ScalarValue { type: Int32, val: Null }` returns `Some(_)`, whereas I expected `None`, indicating that the values are not comparable, consistent with SQL’s three-valued logic.
Additionally, comparing `Null` with `Null` returns `Some(Ordering::Equal)` instead of `None`.
We should either:
1. Fix this behavior to follow SQL’s null comparison semantics, or
2. If this non-standard null behavior is intentional for specific reasons, explicitly document it in the comments.
### To Reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start in datafusion/common/src/scalar/mod.rs at the partial_cmp implementation referenced in the issue, then review the surrounding tests and comments. Add the proposed scalar_partial_ordering_nulls test to observe the current behavior, and confirm whether the intended result is SQL-style incomparability or documented existing behavior before making the issue's outcome explicit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100