Stop retrying downloading if a segment if zk metadata is missing
Open
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
During the investigation described in https://github.com/apache/pinot/pull/10804, we found the download code in SegmentFetcher would still retry even if the segment metadata is missing, this could make the thread to get held up wastefully. We should add a piece of code to check on the existence of the segment in zk before every time we retry.
Contributor guide
Research direction
Start in the SegmentFetcher download and retry path, using the investigation in pull request 10804 as context. Trace how segment metadata is checked in ZooKeeper before each retry. Done means retries stop when the segment metadata is missing, while normal retries remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100