apache / apache/lucene

Indexwriter changecount assertion fail with g1gc [LUCENE-6098]

Open
#7,160 7 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

This failed on 1.8.0u25 G1GC and I was able to reproduce at least once, but not without G1.

```
svn co -r 1643097 https://svn.eu.apache.org/repos/asf/lucene/dev/trunk
cd trunk/lucene/core
ant beast -Dtestcase=TestStressDeletes -Dtests.method=test -Dtests.seed=C8F513C39231BFA2 -Dtests.slow=true -Dtests.locale=sk -Dtests.timezone=Singapore -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 -Dbeast.iters=1000 -Dargs="-XX:+UseG1GC" -Dtests.dups=8
```

I am not sure this is possible to debug, but the exception is scary:
```
[junit4] Suite: org.apache.lucene.index.TestStressDeletes
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestStressDeletes -Dtests.method=test -Dtests.seed=C8F513C39231BFA2 -Dtests.slow=true -Dtests.locale=sk -Dtests.timezone=Singapore -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.62s J5 | TestStressDeletes.test <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=68, name=Thread-30, state=RUNNABLE, group=TGRP-TestStressDeletes]
[junit4] > at __randomizedtesting.SeedInfo.seed([C8F513C39231BFA2:40A12C193CCDD25A]:0)
[junit4] > Caused by: java.lang.AssertionError: lastCommitChangeCount=130 changeCount=128
[junit4] > at __randomizedtesting.SeedInfo.seed([C8F513C39231BFA2]:0)
[junit4] > at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4256)
[junit4] > at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2778)
[junit4] > at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2881)
[junit4] > at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2848)
[junit4] > at org.apache.lucene.index.TestStressDeletes$1.run(TestStressDeletes.java:75)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene50): {id=PostingsFormat(name=Direct)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {id=DFR I(ne)B1}, locale=sk, timezone=Singapore
[junit4] 2> NOTE: Linux 3.13.0-35-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=8,threads=1,free=428019792,total=499646464
[junit4] 2> NOTE: All tests run in this JVM: [TestOmitNorms, Test2BPositions, Nested2, Nested3, Nested1, Test2BPagedBytes, TestTermsEnum2, TestDoc, TestTwoPhaseCommitTool, TestTimSorter, TestDocTermOrdsRangeFilter, TestWorstCaseTestBehavior, TestBasics, TestToken, TestStressDeletes]
[junit4] Completed on J5 in 0.70s, 1 test, 1 error <<< FAILURES!
```
{noformat}

---
Migrated from [LUCENE-6098](https://issues.apache.org/jira/browse/LUCENE-6098) by Robert Muir (@rmuir), updated Sep 15 2018
Attachments: [LUCENE-6098.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-6098/LUCENE-6098.patch)

Contributor guide

Open the contributing guide

Research direction

Start with TestStressDeletes.java, especially the commit loop around line 75, and follow the assertion in IndexWriter.java at startCommit. Reproduce with the provided ant beast command and G1GC settings, then trace why lastCommitChangeCount differs from changeCount. Done means the failure is understood and the assertion no longer fails under the reported conditions.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.