After segment purge, the segment start and end time in segment zk metadata may not reflect the correct values
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
During the process of purging segments based on a record matcher, certain rows may be eliminated. Once the segment is rebuilt, new metadata is generated. However, in certain scenarios, the time metadata from the previous segment is carried over to the new segment. Consequently, even if rows from the beginning of the segments are removed, the segment start time metadata will still reflect the start time of the segment before purge. The same applies to the segment end time metadata.
Contributor guide
Research direction
Start by tracing the segment purge and rebuild flow that produces segment ZK metadata, focusing on how start and end times are carried from the previous segment. Reproduce a purge that removes rows at either boundary and inspect the resulting metadata; done means both values reflect the rebuilt segment's remaining rows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100