Possible Race Condition with Consistent push clobbers segment lineage
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
If there is an empty table and multiple jobs run start/end replace segments in parallel, there appears to be a reproducible race condition where each job creates the segment lineage entry in zookeeper but the earlier entriy is deleted, so segments appear instantly and end replace segments fails (because of missing lineage id).
To reproduce:
Create 2 segments, then call start replace segments in parallel, upload the segments, call end replace segments.
Contributor guide
Research direction
Start with the reported reproduction: run start/end replace segments in parallel on an empty table, with segment uploads between them. Trace the segment lineage entries created and deleted in ZooKeeper during the parallel start-replace calls. Done means lineage is preserved and end replace segments succeeds without missing lineage IDs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100