apache / apache/lucene

TestFieldSkew test failure [LUCENE-6753]

Open
#7,811 3 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

This test adds empty documents with no field contents, then reopens and asserts that scores are the same. Its not supposed to be a random test, it just does a simple execution against each query.

But it fails with this seed in trunk (i committed some improved debugging):
```
ant test -Dtestcase=TestSimilarity2 -Dtests.method=testNoFieldSkew -Dtests.seed=216F1F3C38561BEF -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.locale=es_ES -Dtests.timezone=America/Catamarca -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] > Throwable #1: java.lang.AssertionError: DefaultSimilarity: actual=0.0 = No matching clauses
[junit4] > ,expected=0.2169777 = sum of:
[junit4] > 0.10848885 = weight(foo:bar in 0) [DefaultSimilarity], result of:
[junit4] > 0.10848885 = score(doc=0,freq=1.0), product of:
[junit4] > 0.70710677 = queryWeight, product of:
[junit4] > 0.30685282 = idf(docFreq=1, docCount=1)
[junit4] > 2.3043842 = queryNorm
[junit4] > 0.15342641 = fieldWeight in 0, product of:
[junit4] > 1.0 = tf(freq=1.0), with freq of:
[junit4] > 1.0 = termFreq=1.0
[junit4] > 0.30685282 = idf(docFreq=1, docCount=1)
[junit4] > 0.5 = fieldNorm(doc=0)
[junit4] > 0.10848885 = weight(foo:baz in 0) [DefaultSimilarity], result of:
[junit4] > 0.10848885 = score(doc=0,freq=1.0), product of:
[junit4] > 0.70710677 = queryWeight, product of:
[junit4] > 0.30685282 = idf(docFreq=1, docCount=1)
[junit4] > 2.3043842 = queryNorm
[junit4] > 0.15342641 = fieldWeight in 0, product of:
[junit4] > 1.0 = tf(freq=1.0), with freq of:
[junit4] > 1.0 = termFreq=1.0
[junit4] > 0.30685282 = idf(docFreq=1, docCount=1)
[junit4] > 0.5 = fieldNorm(doc=0)
[junit4] > expected:<0.21697770059108734> but was:<0.0>
[junit4] > at __randomizedtesting.SeedInfo.seed([216F1F3C38561BEF:89FA6BFD1F67BF87]:0)
[junit4] > at org.apache.lucene.search.similarities.TestSimilarity2.testNoFieldSkew(TestSimilarity2.java:201)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=16060, maxDocsPerChunk=1, blockSize=6), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=16060, blockSize=6)), sim=DefaultSimilarity, locale=es_ES, timezone=America/Catamarca
[junit4] 2> NOTE: Linux 3.19.0-25-generic amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=8,threads=1,free=140426184,total=189267968
[junit4] 2> NOTE: All tests run in this JVM: [TestSimilarity2]
[junit4] Completed [1/1] in 0.64s, 1 test, 1 failure <<< FAILURES!
```

This is scary, it means the first reopen did not make documents visible when it should have? (or there is a sneaky test bug)

---
Migrated from [LUCENE-6753](https://issues.apache.org/jira/browse/LUCENE-6753) by Robert Muir (@rmuir)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing TestSimilarity2.testNoFieldSkew at TestSimilarity2.java:201 with the reported seed and test command. Trace the first reopen and document visibility to determine whether the failure is in reopening or the test, then rerun the focused test with the failure resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.