waitForMerges deadlocks if background merge fails [LUCENE-3373]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
waitForMerges can deadlock if a merge fails for ConcurrentMergeScheduler
this is because the merge thread will die, but pending merges are still available
normally, the merge thread will pick up the next merge once it finishes the previous merge, but in the event of a merge exception, the pending work is not resumed, but waitForMerges won't complete until all pending work is complete
i worked around this by overriding doMerge() like so:
```Java
protected final void doMerge(MergePolicy.OneMerge merge) throws IOException {
try {
super.doMerge(merge);
} catch (Throwable exc) {
// Just logging the exception and not rethrowing
// insert logging code here
}
}
```
Here's the rough steps i used to reproduce this issue:
override doMerge like so
```Java
protected final void doMerge(MergePolicy.OneMerge merge) throws IOException {
try {Thread.sleep(500L);} catch (InterruptedException e) { }
super.doMerge(merge);
throw new IOException("fail");
}
```
then, if you do the following:
loop 50 times:
addDocument // any doc
commit
waitForMerges // This will deadlock sometimes
[SOLR-2017](https://issues.apache.org/jira/browse/SOLR-2017) may be related to this (stack trace for deadlock looked related)
---
Migrated from [LUCENE-3373](https://issues.apache.org/jira/browse/LUCENE-3373) by Tim Smith, 2 votes, updated May 23 2022
Contributor guide
Research direction
Start with ConcurrentMergeScheduler, its doMerge path, and waitForMerges; reproduce the issue using the supplied failing doMerge override, 50 addDocument/commit iterations, and waitForMerges. Trace what happens to pending merges after the merge thread throws, and verify that waitForMerges completes while pending work is resumed or reported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100