apache / apache/arrow

[C++][Compute] Crash of out-of-bounds memory accessing in KeyCompare::NullUpdateColumnToRowImp_avx2 of hash_aggregate

Open
#31,059 1 comment 0 reactions 0 assignees View on GitHub
Component: C++ Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

Run `RandomArraySum` multiple times, there is a high probability that a crash of out-of-bounds memory accessing will occur. The crash is caused by the following code:
```c++

left = _mm256_cvtepu8_epi32(_mm_set1_epi8(static_cast(
reinterpret_cast(non_nulls + i)[0] >> col.bit_offset(0))));
```
This code access the last bytes in the type of `uint16_t`, so one extra byte of null_bitmap is accessed.

`non_nulls` comes from the null_bitmap of an array. If the length of the array is close to or equal to a multiple of 64, then the null_bitmap lacks padding bytes after the last bytes. This code leads to a crash of out-of-bounds memory accessing.

**Reporter**: [Chenxi Li](https://issues.apache.org/jira/browse/ARROW-15597) / @Crystrix

**Note**: *This issue was originally created as [ARROW-15597](https://issues.apache.org/jira/browse/ARROW-15597). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by running RandomArraySum repeatedly with array lengths near or equal to multiples of 64, then inspect KeyCompare::NullUpdateColumnToRowImp_avx2 and the shown null_bitmap access. Done means the reproduction no longer crashes and the access does not read beyond the null bitmap.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data-engineering, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.