Segments not properly deleted when dropping then recreating a LLC table
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
When dropping then recreating a relatively large LLC table, we have observed several issues:
- `HelixInstanceDataManager.deleteTable()` will time out because external view is recreated, thus will never satisfy the check
- Old consuming segment is removed from current state and external view, but never released (seems it is somehow referenced)
- It falls into infinite holding loop under `Holding after response from Controller: {"offset":-1,"buildTimeSec":-1,"isSplitCommitType":false,"status":"FAILED","streamPartitionMsgOffset":null}`
- It blocks other segments for this partition to be consumed/loaded
Contributor guide
Research direction
Start with HelixInstanceDataManager.deleteTable() and trace the external-view recreation and old consuming-segment release paths. Reproduce dropping and recreating a large LLC table, then verify that deletion does not time out, the old segment is released, and other segments are no longer blocked by the holding loop.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100