apache / apache/lucene

JDK 9: TestIndexWriterMerging.testForceMergeDeletes fails with impossible (?) assert trip [LUCENE-6642]

Open
#7,700 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

I suspect there is a JVM bug in JDK 9 here... in this code I don't see how current can be null at this point.

Build failure is here: http://build-eu-00.elastic.co/job/lucene_linux_java8_64_test_only/53666/console

```
[junit4] Suite: org.apache.lucene.index.TestIndexWriterMerging
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterMerging -Dtests.method=testForceMergeDeletes -Dtests.seed=D33F5440CCAADDDB -Dtests.slow=true -Dtests.locale=vi -Dtests.timezone=Pacific/Enderbury -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.16s J0 | TestIndexWriterMerging.testForceMergeDeletes <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at __randomizedtesting.SeedInfo.seed([D33F5440CCAADDDB:E14F04E6F3205DF4]:0)
[junit4] > at org.apache.lucene.util.fst.BytesStore.writeBytes(BytesStore.java:104)
[junit4] > at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
[junit4] > at org.apache.lucene.util.fst.FST.(FST.java:344)
[junit4] > at org.apache.lucene.util.fst.FST.(FST.java:323)
[junit4] > at org.apache.lucene.codecs.blocktreeords.OrdsFieldReader.(OrdsFieldReader.java:83)
[junit4] > at org.apache.lucene.codecs.blocktreeords.OrdsBlockTreeTermsReader.(OrdsBlockTreeTermsReader.java:147)
[junit4] > at org.apache.lucene.codecs.blocktreeords.BlockTreeOrdsPostingsFormat.fieldsProducer(BlockTreeOrdsPostingsFormat.java:93)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:261)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
[junit4] > at org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:104)
[junit4] > at org.apache.lucene.index.SegmentReader.(SegmentReader.java:65)
[junit4] > at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:58)
[junit4] > at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:50)
[junit4] > at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:731)
[junit4] > at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:73)
[junit4] > at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
[junit4] > at org.apache.lucene.index.TestIndexWriterMerging.testForceMergeDeletes(TestIndexWriterMerging.java:151)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene53): {tvtest=FST50, count=PostingsFormat(name=Memory doPackFST= false), id=PostingsFormat(name=Memory doPackFST= false), termVector=BlockTreeOrds(blocksize=128)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {tvtest=DFR I(n)B3(800.0), id=DFR I(n)L2, termVector=DFR GLZ(0.3)}, locale=vi, timezone=Pacific/Enderbury
[junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.9.0-ea (64-bit)/cpus=8,threads=1,free=132149080,total=307232768
[junit4] 2> NOTE: All tests run in this JVM: [TestBytesRef, TestDateSort, TestTermsEnum2, TestIOUtils, TestDocBoost, TestIndexWriterConfig, TestTimeLimitingCollector, TestMultiValuedNumericRangeQuery, TestBagOfPostings, TestConsistentFieldNumbers, TestFieldCacheRewriteMethod, TestFieldType, TestDuelingCodecs, TestSloppyMath, TestMathUtil, TestReusableStringReader, TestSloppyPhraseQuery2, TestTerm, TestExitableDirectoryReader, TestSizeBoundedForceMerge, TestInPlaceMergeSorter, TestLongPostings, TestTermRangeFilter, TestCachingCollector, TestSameTokenSamePosition, TestIndexWriterMerging]
[junit4] Completed [167/402] on J0 in 10.39s, 6 tests, 1 failure <<< FAILURES!
```

---
Migrated from [LUCENE-6642](https://issues.apache.org/jira/browse/LUCENE-6642) by Michael McCandless (@mikemccand)

Contributor guide

Open the contributing guide

Research direction

Reproduce TestIndexWriterMerging.testForceMergeDeletes with the ant command and seed shown in the issue. Trace the failure through BytesStore.writeBytes, DataOutput.copyBytes, FST, and the blocktreeords readers, then determine the cause of the assertion failure and whether the test passes after the issue is addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.