[C++] Handle only relevant slices of child arrays when hashing scalars from ListArrays
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the enhancement requested
Issue is explained in the Python code below:
```python
import pyarrow as pa
a = pa.array([
[{'a': 5}, {'a': 6}],
[{'a': 7}, None]
])
b = pa.array([
[{'a': 7}, None]
])
# a[1] and b[0] are represented as 2-element slices of a child array containing struct values
# they start on different offsets, but obviously compare as equal
assert a[1] == b[0]
# logically equal values should hash to the same value, so when hashing the hashing
# of the child array should start at the offset and not from 0 as it's done by default.
hash1 = hash(a[1])
hash2 = hash(b[0])
assert hash1 == hash2
```
#35814 fixes the bug for lists of structs, but the same bug might exist for other nested types:
- [x] struct
- [ ] sparse union
- [ ] dense union
- [ ] run-end encoded
- [ ] more?
### Component(s)
C++
Contributor guide
Research direction
Start with the Python reproducer and compare the behavior with the lists-of-structs fix referenced in #35814. Trace the C++ scalar hashing path for nested arrays, then check sparse union, dense union, and run-end encoded values. Done means logically equal slices produce identical hashes across the remaining relevant nested types.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100