apache / apache/paimon-cpp

[Feature] Unpack predicate boolean kernel bitmaps a byte at a time

已關閉
#335 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
enhancement
主要語言
C++
星號
65
分支
25
平均合併
2 天 9 小時
30 天內合併 PR
82

描述

## Search before asking

- [x] I searched in the [issues](https://github.com/apache/paimon-cpp/issues) and found nothing similar.

## Motivation

Two predicate leaf functions evaluate a batch by running an `arrow::compute` kernel and reading back the `arrow::BooleanArray` it writes: `MultiLiteralsLeafFunction` (`IN` / `NOT IN`, via `IsIn`) and `NullFalseLeafBinaryFunction` (the comparison functions). A kernel returns a bitmap, one bit per row, but `LeafFunction::Test` returns `std::vector`, one byte per row, so both call sites spread the bits over bytes with the same per-row loop: test `IsNull`, read `Value`, apply the negation `NOT IN` needs, store a byte. That is a shift, a mask and a byte store per row, duplicated across the two call sites, on the selection path every filtered batch goes through.

## Solution

Extract the spread into one helper, `ArrowUtils::UnpackBooleansToBytes(array, negate)`, and read the bitmap a byte at a time instead of a bit at a time:

- A compile-time table maps each of the 256 bitmap bytes to the eight bytes it expands to, so the aligned body produces eight rows per iteration with one lookup and one 8-byte store.
- A scalar head and tail cover the rows sharing a partial leading or trailing byte, which is where the array offset is not byte-aligned; a batch a kernel has just written is aligned, so it takes the fast body throughout.
- The offset and the validity bitmap are honoured exactly as `BooleanArray::Value()` and `Array::IsValid()` honour them, and a null row unpacks to 0 whatever the value bitmap holds for it, which is what both `IN` / `NOT IN` and every `NullFalseLeafBinaryFunction` require.

`MultiLiteralsLeafFunction` passes its `negate` through; `NullFalseLeafBinaryFunction` passes `negate=false`. The bytes each returns are unchanged.

## Anything else?

A property test that asserts the helper equals a row-by-row reference through the very accessors it replaces, over every `(offset, length)` slice of a bitmap whose value and null periods are not multiples of eight and both negate values, pins the offset, validity and negate handling against the definitions it optimizes. No change to any header under `include/paimon/`, the storage format, or the protocol: `ArrowUtils` is an internal utility under `src/paimon/common/utils/arrow/`.

## Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

貢獻指南

開啟貢獻指南

研究方向

從 src/paimon/common/utils/arrow/ 開始,找到 MultiLiteralsLeafFunction 和 NullFalseLeafBinaryFunction,接著透過 LeafFunction::Test 追蹤它們現有的 bitmap-to-byte 迴圈。加入共用的 helper,以及一個涵蓋每個 offset 和 length、兩種 negate 值,以及未按位元組對齊的 validity 區間的 property test。當 helper 與逐列 accessor reference 相符,且兩個 caller 都回傳未變更的 bytes 時,即表示完成。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
cpp
領域
data
Issue 類型
功能
難度
3/5
預估耗時
1-2 天
活躍度
活躍
描述清晰度
描述清楚
新手友好度
74/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。