[C++][Parquet] Reuse memory of passed array in ConvertToDecimal
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the enhancement requested
I am reading an arrow::ListArray of Decimal128 from FixedLengthBinaryArray[16] in parquets and see on flamegraph that RawBytesToDecimalBytes consumes large amount of time and mostly due to page faults.
I assume that the problem happens here
https://github.com/apache/arrow/blob/eb5de184a7e5d02f98526332ace54250417bd232/cpp/src/parquet/arrow/reader_internal.cc#L559-L576
when arrow allocates new buffer for decimals and then parses FLBA into it.
As FLBA type has the same amount of memory per element i guess that this function should reuse given array data.
Possible easy(but very dirty) solution could be static_cast given array to Decimal array (as it does not have additional fields - it wouldn't be an error), const_cast raw_values of it and reverse byte order in-place
Another benefit is reducing memory usage of reading them
### Component(s)
C++, Parquet
Contributor guide
Assessment
This issue has not been assessed yet.