apache / apache/pinot

Segments not properly deleted when dropping then recreating a LLC table

Open
#10,486 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

When dropping then recreating a relatively large LLC table, we have observed several issues:
- `HelixInstanceDataManager.deleteTable()` will time out because external view is recreated, thus will never satisfy the check
- Old consuming segment is removed from current state and external view, but never released (seems it is somehow referenced)
- It falls into infinite holding loop under `Holding after response from Controller: {"offset":-1,"buildTimeSec":-1,"isSplitCommitType":false,"status":"FAILED","streamPartitionMsgOffset":null}`
- It blocks other segments for this partition to be consumed/loaded

Contributor guide

Open the contributing guide

Research direction

Start with HelixInstanceDataManager.deleteTable() and trace the external-view recreation and old consuming-segment release paths. Reproduce dropping and recreating a large LLC table, then verify that deletion does not time out, the old segment is released, and other segments are no longer blocked by the holding loop.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.