apache / apache/lucene

TestStressIndexing2.testMultiConfig() failure: all instances of a given field name must have the same term vectors settings [LUCENE-6941]

Open
#7,999 0 comments 0 reactions 0 assignees View on GitHub
affects-version:5.5 legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

ASF Jenkins found this failure: . I was able to reproduce by beasting (1/10 dups failed on the first beast iteration):

```
[junit4] Suite: org.apache.lucene.index.TestStressIndexing2
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestStressIndexing2 -Dtests.method=testMultiConfig -Dtests.seed=337C1E3DCE453481 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=hr_HR -Dtests.timezone=Asia/Shanghai -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 1.92s J1 | TestStressIndexing2.testMultiConfig <<<
[junit4] > Throwable #1: java.lang.IllegalArgumentException: all instances of a given field name must have the same term vectors settings (storeTermVectorOffsets changed for field="f0")
[junit4] > at __randomizedtesting.SeedInfo.seed([337C1E3DCE453481:FEEE68E563132BCF]:0)
[junit4] > at org.apache.lucene.index.TermVectorsConsumerPerField.start(TermVectorsConsumerPerField.java:170)
[junit4] > at org.apache.lucene.index.TermsHashPerField.start(TermsHashPerField.java:292)
[junit4] > at org.apache.lucene.index.FreqProxTermsWriterPerField.start(FreqProxTermsWriterPerField.java:74)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:611)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:344)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:300)
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
[junit4] > at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
[junit4] > at org.apache.lucene.index.TestStressIndexing2.indexSerial(TestStressIndexing2.java:250)
[junit4] > at org.apache.lucene.index.TestStressIndexing2.testMultiConfig(TestStressIndexing2.java:106)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Suppressed: java.lang.IllegalStateException: close() called in wrong state: RESET
[junit4] > at org.apache.lucene.analysis.MockTokenizer.fail(MockTokenizer.java:126)
[junit4] > at org.apache.lucene.analysis.MockTokenizer.close(MockTokenizer.java:293)
[junit4] > at org.apache.lucene.analysis.TokenFilter.close(TokenFilter.java:58)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:687)
[junit4] > ... 44 more
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestStressIndexing2_337C1E3DCE453481-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene54): {f25=FSTOrd50, f32=FSTOrd50, id=PostingsFormat(name=LuceneVarGapDocFreqInterval), f19=Lucene50(blocksize=128), f68=Lucene50(blocksize=128), f43=FSTOrd50, f15=Lucene50(blocksize=128), f14=FSTOrd50, f95=Lucene50(blocksize=128), f33=Lucene50(blocksize=128), f65=FSTOrd50, f93=Lucene50(blocksize=128), f77=Lucene50(blocksize=128), f12=PostingsFormat(name=LuceneVarGapDocFreqInterval), f71=Lucene50(blocksize=128), f83=FSTOrd50, f91=Lucene50(blocksize=128), f2=Lucene50(blocksize=128), f40=Lucene50(blocksize=128), f10=FSTOrd50, f28=Lucene50(blocksize=128), f92=PostingsFormat(name=LuceneVarGapDocFreqInterval), f81=PostingsFormat(name=LuceneVarGapDocFreqInterval), f57=Lucene50(blocksize=128), f22=Lucene50(blocksize=128), f58=FSTOrd50, f23=PostingsFormat(name=LuceneVarGapDocFreqInterval), f13=Lucene50(blocksize=128), f46=Lucene50(blocksize=128), f48=Lucene50(blocksize=128), f82=Lucene50(blocksize=128), f36=FSTOrd50, f52=PostingsFormat(name=LuceneVarGapDocFreqInterval), f31=Lucene50(blocksize=128), f27=PostingsFormat(name=LuceneVarGapDocFreqInterval), f90=FSTOrd50, f69=FSTOrd50, f80=Lucene50(blocksize=128), f73=Lucene50(blocksize=128), f17=Lucene50(blocksize=128), f1=PostingsFormat(name=LuceneVarGapDocFreqInterval), f86=Lucene50(blocksize=128), f63=PostingsFormat(name=LuceneVarGapDocFreqInterval), f26=Lucene50(blocksize=128), f64=Lucene50(blocksize=128), f51=Lucene50(blocksize=128), f84=Lucene50(blocksize=128), f5=PostingsFormat(name=LuceneVarGapDocFreqInterval), f8=Lucene50(blocksize=128), f49=PostingsFormat(name=LuceneVarGapDocFreqInterval), f20=Lucene50(blocksize=128), f29=FSTOrd50, f59=Lucene50(blocksize=128), f39=Lucene50(blocksize=128), f9=PostingsFormat(name=LuceneVarGapDocFreqInterval), f55=Lucene50(blocksize=128), f94=FSTOrd50, f99=Lucene50(blocksize=128), f74=PostingsFormat(name=LuceneVarGapDocFreqInterval), f47=FSTOrd50, f41=PostingsFormat(name=LuceneVarGapDocFreqInterval), f37=Lucene50(blocksize=128), f98=FSTOrd50, f21=FSTOrd50, f42=Lucene50(blocksize=128), f62=Lucene50(blocksize=128), f89=PostingsFormat(name=LuceneVarGapDocFreqInterval), f7=FSTOrd50, f67=PostingsFormat(name=LuceneVarGapDocFreqInterval), f78=PostingsFormat(name=LuceneVarGapDocFreqInterval), f24=Lucene50(blocksize=128), f56=PostingsFormat(name=LuceneVarGapDocFreqInterval), f76=FSTOrd50, f0=Lucene50(blocksize=128), f38=PostingsFormat(name=LuceneVarGapDocFreqInterval), f53=Lucene50(blocksize=128), f16=PostingsFormat(name=LuceneVarGapDocFreqInterval), f72=FSTOrd50, f54=FSTOrd50, f45=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{}, sim=DefaultSimilarity, locale=hr_HR, timezone=Asia/Shanghai
[junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=199000432,total=376963072
[junit4] 2> NOTE: All tests run in this JVM: [TestNRTCachingDirectory, TestCodecs, TestIntArrayDocIdSet, TestPayloads, TestFastDecompressionMode, TestScoreCachingWrappingScorer, TestDateTools, TestIntBlockPool, TestIndexWriterForceMerge, TestUnicodeUtil, TestIndexWriterDeleteByQuery, TestBinaryDocument, TestCachingTokenFilter, TestLongBitSet, TestAddIndexes, TestDisjunctionMaxQuery, TestLucene50TermVectorsFormat, TestMergeSchedulerExternal, Test2BNumericDocValues, TestMultiCollector, TestIndexWriterMaxDocs, TestDirectoryReaderReopen, TestFieldReuse, TestDocument, TestSpanExplanationsOfNonMatches, TestSameScoresWithThreads, TestOmitPositions, TestBytesRefArray, TestDuelingCodecsAtNight, TestTermVectors, TestWeakIdentityMap, TestTermdocPerf, TestSegmentMerger, TestNIOFSDirectory, TestNewestSegment, TestNeedsScores, TestNotDocIdSet, TestDemoParallelLeafReader, TestBooleanMinShouldMatch, TestPositiveScoresOnlyCollector, TestFieldMaskingSpanQuery, TestSpanSearchEquivalence, TestFieldValueQuery, TestFilteredSearch, TestFieldType, TestForceMergeForever, TestFixedBitDocIdSet, TestStressIndexing2]
[junit4] Completed [91/414 (1!)] on J1 in 2.28s, 3 tests, 1 error <<< FAILURES!
```

---
Migrated from [LUCENE-6941](https://issues.apache.org/jira/browse/LUCENE-6941) by Steven Rowe (@sarowe)

Contributor guide

Open the contributing guide

Research direction

Start with lucene/core/src/test/org/apache/lucene/index/TestStressIndexing2.java, especially testMultiConfig() and indexSerial(), then reproduce using the listed ant command and seed. Trace the failure from TermVectorsConsumerPerField.start() through DefaultIndexingChain. Done means the reported TestStressIndexing2.testMultiConfig failure is understood and the test passes reliably under the reproducing configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.