MergeScheduler should not implement Closeable [LUCENE-5885]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
MergeScheduler implements Closeable and IndexWriter calls ms.close() when it's closed. But MergeScheduler can be shared between several writers, which means closing it by any particular writer is wrong. We should rather implement some ref-counting logic such that each IW will call incRef() in the ctor, and decRef() on close(), and MergeScheduler will truly close when the ref-count hits 0.
As it is now, if you share a MergeScheduler between writers and close() does something terminating, I doubt if it really works.
Also, when I look at ConcurrentMergeScheduler.close(), it calls sync() which joins all MergeThreads. But if that CMS instance is shared between few IWs, doesn't it mean that a single IW calling close() waits on MergeThreads that execute merges of other IWs!?!? This seems ... wrong?
---
Migrated from [LUCENE-5885](https://issues.apache.org/jira/browse/LUCENE-5885) by Shai Erera (@shaie)
Attachments: [LUCENE-5885.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5885/LUCENE-5885.patch) (versions: 2)
Contributor guide
Research direction
Start by reading MergeScheduler, IndexWriter, and ConcurrentMergeScheduler, focusing on the close and sync interactions described in the issue. Review the attached LUCENE-5885.patch for prior context. Done means shared schedulers remain usable until their last writer closes, without one writer waiting on or terminating merges belonging to others.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100