apache / apache/druid

Segment dropped from info_dir directory but not from segmentCache

Open
#15,764 0 comments 0 reactions 0 assignees View on GitHub
Area - Cache
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

I marked all segments of a datasource as unused. I see that the coordinator has issued a drop segment request to the historical.
I can see log like "Dropping uneeded segment" for that particular segment, for that particular historical in the coordinator logs, but i m not seeing any "unannouncing segment" log in the historical for the same, although, the segment is deleted from the info_dir directory in that particular historical.

### Affected Version
druid 0.22.1
### Description

Please include as much detailed information about the problem as possible.
- I have 2 coordinators and 150 historicals.
- My segments are sharded. I can see some shards of that particular segments getting dropped from that historical but not all of them.

Can you please help me debug this situation or the scenarios in which these might occur.
I believe, if there had been any communication / network issues, the segment should not have been dropped from the info_dir directory, but here, it is deleted from one but not the other.

Contributor guide

Open the contributing guide

Research direction

Start by comparing the coordinator's drop-segment logs with the historical's info_dir and segmentCache state for sharded segments. Reproduce the discrepancy across the reported coordinator and historical setup, then determine the conditions under which a segment is removed from info_dir but remains cached; done means the behavior is explained and covered by an appropriate regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.