Topic deletion in multi topic Improvements
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
Topics are allowed to be deleted in multi topic setups. Safe deletion of segments only happens when you pause the topic first before removing the topic from the config. Pausing the topic forces all CONSUMING segments to go into ONLINE state. These are cleaned up as normal by the RetentionManager.
If the topic was removed from the config during update without pausing, the consuming segments are not getting cleaned up by the RetentionManager. This is because they stay assigned to servers in Helix, causing the deletion to be postponed repeatedly. We keep the [last completed segments](https://github.com/apache/pinot/blob/c899956b/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java#L1002) ([caller](https://github.com/apache/pinot/blob/c899956b/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/retention/RetentionManager.java#L228C1-L229C1)).
I see 2 types of lingering segments
When a topic is deleted from config without pausing, I see 2 types of lingering segments, 1. empty DONE segments (0 docs) 2. BAD state segments that have props like
{ "segment.creation.time": "1767233011933",
"segment.flush.threshold.size": "2000",
"segment.realtime.startOffset": "0",
"segment.realtime.status": "IN_PROGRESS" }
Potential Solutions
- we could check the config and see the if topic was actually deleted and remove the segments i.e ignore the last consuming status if the topic has been removed.
- Prevent this state from being reached.
- Run pause during config update call if topic is being removed
- OR add a validation to fail update config is to remove topic it is not paused. The validation should be skippable in the normal way if users want.
Contributor guide
Assessment
This issue has not been assessed yet.