TestIndexWriterExceptions.testNoLostDeletesOrUpdates() failure [LUCENE-7724]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Non-reproducing branch_6x seed from my Jenkins:
```
Checking out Revision bce1417fceeed2054f16565e96dc49268c1b2ea1 (refs/remotes/origin/branch_6x)
[...]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterExceptions -Dtests.method=testNoLostDeletesOrUpdates -Dtests.seed=611686EB379EF662 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en-US -Dtests.timezone=America/Cordoba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 9.90s J6 | TestIndexWriterExceptions.testNoLostDeletesOrUpdates <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=343, name=Lucene Merge Thread #21, state=RUNNABLE, group=TGRP-TestIndexWriterExceptions]
[junit4] > Caused by: org.apache.lucene.index.MergePolicy$MergeException: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
[junit4] > at __randomizedtesting.SeedInfo.seed([611686EB379EF662]:0)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
[junit4] > Caused by: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
[junit4] > at org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:345)
[junit4] > at org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:696)
[junit4] > at org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:691)
[junit4] > at org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:4964)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4455)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3940)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
[junit4] > Caused by: org.apache.lucene.store.MockDirectoryWrapper$FakeIOException
[junit4] > at org.apache.lucene.index.TestIndexWriterExceptions$11.eval(TestIndexWriterExceptions.java:1875)
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:1022)
[junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:144)
[junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:126)
[junit4] > at org.apache.lucene.store.DataOutput.writeInt(DataOutput.java:70)
[junit4] > at org.apache.lucene.codecs.CodecUtil.writeHeader(CodecUtil.java:90)
[junit4] > at org.apache.lucene.codecs.CodecUtil.writeIndexHeader(CodecUtil.java:133)
[junit4] > at org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.(Lucene54DocValuesConsumer.java:74)
[junit4] > at org.apache.lucene.codecs.lucene54.Lucene54DocValuesFormat.fieldsConsumer(Lucene54DocValuesFormat.java:108)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:213)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:111)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.handleNumericDVUpdates(ReadersAndUpdates.java:328)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:521)
[junit4] > at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:272)
[junit4] > at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4119)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4077)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3931)
[junit4] > ... 2 moreThrowable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=344, name=Lucene Merge Thread #22, state=RUNNABLE, group=TGRP-TestIndexWriterExceptions]
[junit4] > Caused by: org.apache.lucene.index.MergePolicy$MergeException: org.apache.lucene.store.MockDirectoryWrapper$FakeIOException
[junit4] > at __randomizedtesting.SeedInfo.seed([611686EB379EF662]:0)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
[junit4] > Caused by: org.apache.lucene.store.MockDirectoryWrapper$FakeIOException
[junit4] > at org.apache.lucene.index.TestIndexWriterExceptions$11.eval(TestIndexWriterExceptions.java:1875)
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:1022)
[junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:144)
[junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:126)
[junit4] > at org.apache.lucene.store.DataOutput.writeInt(DataOutput.java:70)
[junit4] > at org.apache.lucene.codecs.CodecUtil.writeHeader(CodecUtil.java:90)
[junit4] > at org.apache.lucene.codecs.CodecUtil.writeIndexHeader(CodecUtil.java:133)
[junit4] > at org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.(Lucene54DocValuesConsumer.java:74)
[junit4] > at org.apache.lucene.codecs.lucene54.Lucene54DocValuesFormat.fieldsConsumer(Lucene54DocValuesFormat.java:108)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:213)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:111)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.handleNumericDVUpdates(ReadersAndUpdates.java:328)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:521)
[junit4] > at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:272)
[junit4] > at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4119)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4077)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3931)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false,coord=no): {content6=DFR I(F)BZ(0.3), field=DFR I(n)BZ(0.3), content4=DFR I(F)L1, contents=DFR I(n)1, content2=DFR I(ne)Z(0.3), content1=IB LL-L1, id=DFR I(n)L3(800.0), content=DFR I(F)1}, locale=en-US, timezone=America/Cordoba
[junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 (64-bit)/cpus=16,threads=1,free=421632952,total=456654848
```
---
Migrated from [LUCENE-7724](https://issues.apache.org/jira/browse/LUCENE-7724) by Steven Rowe (@sarowe)
Contributor guide
Research direction
Start with TestIndexWriterExceptions.java around line 1875 and run the supplied Ant command with the recorded seed. Trace the concurrent merge and update failure through the stack frames shown, including IndexWriter and ReadersAndUpdates; done means the intermittent test failure is understood and a correction is covered by the test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100