apache / apache/lucene

TestThreadedForceMerge failure [LUCENE-3817]

Open
#4,890 0 comments 0 reactions 0 assignees View on GitHub
affects-version:4.0-ALPHA legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

leaks a file handle....

was running tests a lot to keep my computer busy while debugging something else.
```
[junit] Testsuite: org.apache.lucene.index.TestThreadedForceMerge
[junit] Testcase: testThreadedForceMerge(org.apache.lucene.index.TestThreadedForceMerge): Caused an ERROR
[junit] MockDirectoryWrapper: cannot close: there are still open files: {_58_1.skp=1, _58_1.tib=1, _58_1.tii=1, _58_0.frq=1, _58_0.tim=1, _58.tvf=1, _58.tvd=1, _58_1.doc=1, _58.tvx=1, _58.fdt=1, _58.fdx=1}
[junit] java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open files: {_58_1.skp=1, _58_1.tib=1, _58_1.tii=1, _58_0.frq=1, _58_0.tim=1, _58.tvf=1, _58.tvd=1, _58_1.doc=1, _58.tvx=1, _58.fdt=1, _58.fdx=1}
[junit] at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:555)
[junit] at org.apache.lucene.index.TestThreadedForceMerge.testThreadedForceMerge(TestThreadedForceMerge.java:141)
[junit] at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:707)
[junit] at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:606)
[junit] at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:511)
[junit] at org.apache.lucene.util.LuceneTestCase$RememberThreadRule$1.evaluate(LuceneTestCase.java:569)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:165)
[junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:57)
[junit] Caused by: java.lang.RuntimeException: unclosed IndexInput: _58_1.doc
[junit] at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:479)
[junit] at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:504)
[junit] at org.apache.lucene.codecs.mocksep.MockSingleIntIndexInput.(MockSingleIntIndexInput.java:41)
[junit] at org.apache.lucene.codecs.mocksep.MockSingleIntFactory.openInput(MockSingleIntFactory.java:32)
[junit] at org.apache.lucene.codecs.sep.SepPostingsReader.(SepPostingsReader.java:68)
[junit] at org.apache.lucene.codecs.mocksep.MockSepPostingsFormat.fieldsProducer(MockSepPostingsFormat.java:89)
[junit] at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader$1.visitOneFormat(PerFieldPostingsFormat.java:189)
[junit] at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$VisitPerFieldFile.(PerFieldPostingsFormat.java:280)
[junit] at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader$1.(PerFieldPostingsFormat.java:186)
[junit] at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:186)
[junit] at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:256)
[junit] at org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:108)
[junit] at org.apache.lucene.index.SegmentReader.(SegmentReader.java:51)
[junit] at org.apache.lucene.index.IndexWriter$ReadersAndLiveDocs.getMergeReader(IndexWriter.java:521)
[junit] at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3586)
[junit] at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3256)
[junit] at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:382)
[junit] at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:451)
[junit]
[junit]
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.704 sec
[junit]
[junit] ------------- Standard Error -----------------
[junit] NOTE: reproduce with: ant test -Dtestcase=TestThreadedForceMerge -Dtestmethod=testThreadedForceMerge -Dtests.seed=-36d98ee49f3f6205:-602830ebcb0e2f6d:9429978db44e64b -Dargs="-Dfile.encoding=UTF-8"
[junit] NOTE: test params are: codec=Lucene40: {id=PostingsFormat(name=MockSep), contents=PostingsFormat(name=NestedPulsing)}, sim=RandomSimilarityProvider(queryNorm=false,coord=true): {}, locale=ar_JO, timezone=Asia/Pontianak
[junit] NOTE: all tests run in this JVM:
[junit] [TestSearchForDuplicates, TestMockAnalyzer, TestToken, TestIntBlockCodec, TestImpersonation, TestBitVector, TestAtomicUpdate, TestBinaryTerms, TestCheckIndex, TestCrashCausesCorruptIndex, TestDirectoryReaderReopen, TestDocCount, TestDocValuesIndexing, TestForTooMuchCloning, TestIndexInput, TestIndexWriter, TestIndexWriterMerging, TestIndexWriterNRTIsCurrent, TestIndexWriterOnDiskFull, TestIndexWriterWithThreads, TestLazyProxSkipping, TestLongPostings, TestOmitTf, TestParallelAtomicReader, TestParallelCompositeReader, TestPayloadProcessorProvider, TestSameTokenSamePosition, TestSegmentTermDocs, TestSnapshotDeletionPolicy, TestSumDocFreq, TestTermVectorsWriter, TestThreadedForceMerge]
[junit] NOTE: Linux 3.0.0-14-generic amd64/Sun Microsystems Inc. 1.6.0_24 (64-bit)/cpus=8,threads=1,free=177836920,total=287309824
[junit] ------------- ---------------- ---------------
```

---
Migrated from [LUCENE-3817](https://issues.apache.org/jira/browse/LUCENE-3817) by Robert Muir (@rmuir)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing TestThreadedForceMerge.testThreadedForceMerge with the provided ant command and inspect TestThreadedForceMerge.java at line 141. Trace the unclosed inputs through MockDirectoryWrapper, MockSingleIntIndexInput, SepPostingsReader, and IndexWriter merge handling. Done means the test passes without open-file errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.