[C++][Acero][Python] Asof join does not detect null times
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
The current asof join implementation does not check the null bitmap before indexing into the time column's value buffer. This creates matches that should not be possible. For example,
```python
import pyarrow as pa
lhs = pa.table({
"time": pa.array([None], type=pa.int64())
})
rhs = pa.table({
"time": [0],
"data": [True],
})
lhs.join_asof(rhs, "time", [], 0)
```
produces
```
time data
0 null true
```
which implies that null equals 0. By manipulating the value buffer, we can get null to "equal" any integer:
```python
lhs = pa.table({
"time": pa.Array.from_buffers(
pa.int32(),
1,
[pa.py_buffer(b"\x00"), pa.py_buffer(b"\xf0\x00\x00\x00")],
),
})
rhs = pa.table({
"time": pa.array([0xf0], type=pa.int32()),
"payload": ["abc"],
})
# lhs: rhs:
# time time payload
# 0 null 0 240 abc
result = lhs.join_asof(rhs, "time", [], 0)
# result:
# time payload
# 0 null abc -> non-null payload means null matched 240
```
This could be fixed by adding a null check in the `GetTime` function in `time_series_util.cc`, but it's not clear to me what the correct behavior should be.
### Component(s)
C++
Contributor guide
Assessment
This issue has not been assessed yet.