[Bug] RTP Tool Should Consider Segment Relocation
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Currently, the RTP tool calculates `numCompletedSegmentsPerPartition` as:
`int numCompletedSegmentsPerPartition = (_tableRetentionHours + numHoursToConsume - 1) / numHoursToConsume - 1;`
`_tableRetentionHours` is read from the user-supplied table config. However, this does not work in the scenario where segments are relocated using tag overrides. For example, we move segments from REALTIME to OFFLINE servers every hour. While the table's retention period is 7d, here, `_tableRetentionHours` should be 1h instead of 7d as completed segments reside in REALTIME servers only for about an hour.
@mcvsubbu
Contributor guide
Research direction
Locate the RTP tool code containing the numCompletedSegmentsPerPartition calculation and read how table retention and segment placement are represented. Reproduce the tag-override scenario described in the issue, then verify that the calculation uses the time segments remain on REALTIME servers rather than the table's overall retention period.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100