[C++][Compute] Crash of out-of-bounds memory accessing in KeyCompare::NullUpdateColumnToRowImp_avx2 of hash_aggregate
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
Run `RandomArraySum` multiple times, there is a high probability that a crash of out-of-bounds memory accessing will occur. The crash is caused by the following code:
```c++
left = _mm256_cvtepu8_epi32(_mm_set1_epi8(static_cast(
reinterpret_cast(non_nulls + i)[0] >> col.bit_offset(0))));
```
This code access the last bytes in the type of `uint16_t`, so one extra byte of null_bitmap is accessed.
`non_nulls` comes from the null_bitmap of an array. If the length of the array is close to or equal to a multiple of 64, then the null_bitmap lacks padding bytes after the last bytes. This code leads to a crash of out-of-bounds memory accessing.
**Reporter**: [Chenxi Li](https://issues.apache.org/jira/browse/ARROW-15597) / @Crystrix
**Note**: *This issue was originally created as [ARROW-15597](https://issues.apache.org/jira/browse/ARROW-15597). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start by running RandomArraySum repeatedly with array lengths near or equal to multiples of 64, then inspect KeyCompare::NullUpdateColumnToRowImp_avx2 and the shown null_bitmap access. Done means the reproduction no longer crashes and the access does not read beyond the null bitmap.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data-engineering, performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 38/100