TestIndexWriterOnDiskFull.testAddDocumentOnDiskFull() failure [LUCENE-6726]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
My Linux Jenkins found a seed that reproduces 100% for me on OS X w/Java7, but only if I run the whole suite (i.e., exclude `-Dtests.method=testAddDocumentOnDiskFull` from the repro line) :
```
[junit4:pickseed] Seed property 'tests.seed' already defined: 6B36AD57BB5A1779
[junit4] says שלום! Master seed: 6B36AD57BB5A1779
[junit4] Executing 1 suite with 1 JVM.
[junit4]
[junit4] Started J0 PID(86114@localhost).
[junit4] Suite: org.apache.lucene.index.TestIndexWriterOnDiskFull
[junit4] HEARTBEAT J0 PID(86114@localhost): 2015-08-06T09:44:03, stalled for 70.7s at: TestIndexWriterOnDiskFull.testAddIndexOnDiskFull
[junit4] OK 73.8s | TestIndexWriterOnDiskFull.testAddIndexOnDiskFull
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterOnDiskFull -Dtests.method=testAddDocumentOnDiskFull -Dtests.seed=6B36AD57BB5A1779 -Dtests.nightly=true -Dtests.slow=true -Dtests.locale=ar_AE -Dtests.timezone=America/Boa_Vista -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.25s | TestIndexWriterOnDiskFull.testAddDocumentOnDiskFull <<<
[junit4] > Throwable #1: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
[junit4] > at __randomizedtesting.SeedInfo.seed([6B36AD57BB5A1779:E72E952846CFC1BD]:0)
[junit4] > at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:719)
[junit4] > at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:733)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1471)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
[junit4] > at org.apache.lucene.index.TestIndexWriterOnDiskFull.addDoc(TestIndexWriterOnDiskFull.java:575)
[junit4] > at org.apache.lucene.index.TestIndexWriterOnDiskFull.testAddDocumentOnDiskFull(TestIndexWriterOnDiskFull.java:80)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: java.io.IOException: fake disk full at 7402 bytes when writing _4_Lucene50_0.dvm (file length=0)
[junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.checkDiskFull(MockIndexOutputWrapper.java:88)
[junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:134)
[junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:127)
[junit4] > at org.apache.lucene.store.RateLimitedIndexOutput.writeByte(RateLimitedIndexOutput.java:66)
[junit4] > at org.apache.lucene.store.DataOutput.writeInt(DataOutput.java:70)
[junit4] > at org.apache.lucene.codecs.CodecUtil.writeHeader(CodecUtil.java:91)
[junit4] > at org.apache.lucene.codecs.CodecUtil.writeIndexHeader(CodecUtil.java:134)
[junit4] > at org.apache.lucene.codecs.lucene50.Lucene50DocValuesConsumer.(Lucene50DocValuesConsumer.java:68)
[junit4] > at org.apache.lucene.codecs.lucene50.Lucene50DocValuesFormat.fieldsConsumer(Lucene50DocValuesFormat.java:197)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:187)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:111)
[junit4] > at org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:252)
[junit4] > at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:163)
[junit4] > at org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:150)
[junit4] > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:105)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4089)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3664)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
[junit4] OK 0.12s | TestIndexWriterOnDiskFull.testCorruptionAfterDiskFullDuringMerge
[junit4] OK 0.00s | TestIndexWriterOnDiskFull.testImmediateDiskFull
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/sarowe/svn/lucene/dev/branches/branch_5x/lucene/build/core/test/J0/temp/lucene.index.TestIndexWriterOnDiskFull_6B36AD57BB5A1779-002
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene53): {id=PostingsFormat(name=MockRandom), f=PostingsFormat(name=Memory doPackFST= false), content=PostingsFormat(name=SimpleText)}, docValues:{numericdv=DocValuesFormat(name=Lucene50)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {id=LM Jelinek-Mercer(0.700000), f=DFR I(ne)3(800.0), content=DFR GL2}, locale=ar_AE, timezone=America/Boa_Vista
[junit4] 2> NOTE: Mac OS X 10.10.4 x86_64/Oracle Corporation 1.7.0_71 (64-bit)/cpus=8,threads=1,free=159451480,total=211288064
[junit4] 2> NOTE: All tests run in this JVM: [TestIndexWriterOnDiskFull]
[junit4] Completed [1/1] in 74.53s, 4 tests, 1 error <<< FAILURES!
```
Not sure if they're related, but the most recent changes to this test were made under #7637 and #7674.
---
Migrated from [LUCENE-6726](https://issues.apache.org/jira/browse/LUCENE-6726) by Steven Rowe (@sarowe)
Contributor guide
Research direction
Start with TestIndexWriterOnDiskFull.testAddDocumentOnDiskFull() and its addDoc helper at TestIndexWriterOnDiskFull.java:80 and :575, then inspect the IndexWriter and MockIndexOutputWrapper stack frames. Run the supplied ant command with seed 6B36AD57BB5A1779, including the whole-suite condition. Done means the reproduced disk-full test no longer fails with AlreadyClosedException and the suite passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100