Move SegmentReducer reduce() method to Off-Heap Implementation
Open
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Serialize the generic row (not the default object serde, but a custom one).
Use a PinotDataBuffer to store those off-heap
Sort the rows in the buffer
Implementation will be similar to OffHeapSingleTreeBuilder’s sortAndAggregateSegmentRecords method.
Contributor guide
Research direction
Locate SegmentReducer and OffHeapSingleTreeBuilder, then read OffHeapSingleTreeBuilder’s sortAndAggregateSegmentRecords method first. Trace how generic rows are serialized and stored in a PinotDataBuffer, and consider the work complete when SegmentReducer.reduce() uses that off-heap path with custom serialization and sorted rows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100