apache / apache/lucene

org.apache.lucene.search.TestSimpleExplanationsWithFillerDocs#testMPQ3 fails reproducible

Open
#12,063 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-label:test-failure type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Policeman Jenkins failed when executing this test on OpenJ9 JVM, but actually it is reproducible:

```
Java: 64bit/openj9/jdk-17.0.5 -XX:-UseCompressedOops -Xgcpolicy:gencon

1 tests failed.
FAILED: org.apache.lucene.search.TestSimpleExplanationsWithFillerDocs.testMPQ3

Error Message:
java.lang.AssertionError: expected:<0.5828427076339722> but was:<0.5828428268432617>

Stack Trace:
java.lang.AssertionError: expected:<0.5828427076339722> but was:<0.5828428268432617>
at __randomizedtesting.SeedInfo.seed([E853C78F22129ACC:708B13FE3EC459FF]:0)
at app//org.junit.Assert.fail(Assert.java:89)
at app//org.junit.Assert.failNotEquals(Assert.java:835)
at app//org.junit.Assert.assertEquals(Assert.java:555)
at app//org.junit.Assert.assertEquals(Assert.java:685)
at app//org.apache.lucene.tests.search.CheckHits.verifyExplanation(CheckHits.java:503)
at app//org.apache.lucene.tests.search.CheckHits.verifyExplanation(CheckHits.java:428)
at app//org.apache.lucene.tests.search.CheckHits.verifyExplanation(CheckHits.java:428)
at app//org.apache.lucene.tests.search.CheckHits$ExplanationAsserter.collect(CheckHits.java:623)
at app//org.apache.lucene.tests.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:52)
at app//org.apache.lucene.tests.search.AssertingCollector$1.collect(AssertingCollector.java:66)
at app//org.apache.lucene.tests.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:52)
at app//org.apache.lucene.tests.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:52)
at app//org.apache.lucene.tests.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:52)
at app//org.apache.lucene.tests.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:52)
at app//org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:282)
at app//org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:254)
at app//org.apache.lucene.tests.search.AssertingBulkScorer.score(AssertingBulkScorer.java:101)
at app//org.apache.lucene.search.ReqExclBulkScorer.score(ReqExclBulkScorer.java:46)
at app//org.apache.lucene.tests.search.AssertingBulkScorer.score(AssertingBulkScorer.java:101)
at app//org.apache.lucene.search.TimeLimitingBulkScorer.score(TimeLimitingBulkScorer.java:76)
at app//org.apache.lucene.search.BulkScorer.score(BulkScorer.java:38)
at app//org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:776)
at app//org.apache.lucene.tests.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:78)
at app//org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:694)
at app//org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:688)
at app//org.apache.lucene.tests.search.CheckHits.checkExplanations(CheckHits.java:336)
at app//org.apache.lucene.tests.search.QueryUtils.checkExplanations(QueryUtils.java:114)
at app//org.apache.lucene.tests.search.QueryUtils.check(QueryUtils.java:144)
at app//org.apache.lucene.tests.search.QueryUtils.check(QueryUtils.java:140)
at app//org.apache.lucene.tests.search.QueryUtils.check(QueryUtils.java:129)
at app//org.apache.lucene.tests.search.CheckHits.checkHitCollector(CheckHits.java:105)
at app//org.apache.lucene.tests.search.BaseExplanationTestCase.qtest(BaseExplanationTestCase.java:110)
at app//org.apache.lucene.search.TestSimpleExplanationsWithFillerDocs.qtest(TestSimpleExplanationsWithFillerDocs.java:116)
at app//org.apache.lucene.search.TestSimpleExplanations.testMPQ3(TestSimpleExplanations.java:229)
at [java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0](mailto:java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0)(Native Method)
at [java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke](mailto:java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke)(NativeMethodAccessorImpl.java:77)
at [java.base@17.0.5/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke](mailto:java.base@17.0.5/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke)(DelegatingMethodAccessorImpl.java:43)
at [java.base@17.0.5/java.lang.reflect.Method.invoke](mailto:java.base@17.0.5/java.lang.reflect.Method.invoke)(Method.java:568)
at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
at app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
at [java.base@17.0.5/java.lang.Thread.run](mailto:java.base@17.0.5/java.lang.Thread.run)(Thread.java:857)
```

Reproduce failure on Java 19 (Hotspot):

```
org.apache.lucene.search.TestSimpleExplanationsWithFillerDocs > test suite's output saved to C:\Users\Uwe Schindler\Projects\lucene\lucene\lucene\core\build\test-results\test\outputs\OUTPUT-org.apache.lucene.search.TestSimpleExplanationsWithFillerDocs.txt, copied below:
2> Jan. 03, 2023 6:30:02 PM org.apache.lucene.store.MemorySegmentIndexInputProvider
2> INFORMATION: Using MemorySegmentIndexInput with Java 19
> java.lang.AssertionError: expected:<0.5828427076339722> but was:<0.5828428268432617>
> at __randomizedtesting.SeedInfo.seed([E853C78F22129ACC:708B13FE3EC459FF]:0)
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotEquals(Assert.java:835)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:685)
> at org.apache.lucene.tests.search.CheckHits.verifyExplanation(CheckHits.java:503)
```

### Version and environment details

Main and 9.x branches

### Reproduce line

gradlew test --tests TestSimpleExplanationsWithFillerDocs.testMPQ3 -Dtests.seed=E853C78F22129ACC -Dtests.multiplier=3 -Dtests.locale=wo -Dtests.timezone=EST5EDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8

Contributor guide

Open the contributing guide

Research direction

Start with org.apache.lucene.search.TestSimpleExplanationsWithFillerDocs.testMPQ3 and its qtest entry point, then run the provided Gradle command with the listed seed. Compare the expected and actual explanation scores and trace the failure through CheckHits.verifyExplanation. Done means the reproducible test passes on the affected JVM environments without weakening the test's coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.