Segment dropped from info_dir directory but not from segmentCache
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
I marked all segments of a datasource as unused. I see that the coordinator has issued a drop segment request to the historical.
I can see log like "Dropping uneeded segment" for that particular segment, for that particular historical in the coordinator logs, but i m not seeing any "unannouncing segment" log in the historical for the same, although, the segment is deleted from the info_dir directory in that particular historical.
### Affected Version
druid 0.22.1
### Description
Please include as much detailed information about the problem as possible.
- I have 2 coordinators and 150 historicals.
- My segments are sharded. I can see some shards of that particular segments getting dropped from that historical but not all of them.
Can you please help me debug this situation or the scenarios in which these might occur.
I believe, if there had been any communication / network issues, the segment should not have been dropped from the info_dir directory, but here, it is deleted from one but not the other.
Contributor guide
Research direction
Start by comparing the coordinator's drop-segment logs with the historical's info_dir and segmentCache state for sharded segments. Reproduce the discrepancy across the reported coordinator and historical setup, then determine the conditions under which a segment is removed from info_dir but remains cached; done means the behavior is explained and covered by an appropriate regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100