TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnCommit fails on nightly
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
This fails with an OOM on nightly. I wonder if we are just merging too much :).
I suspect this is due to the tiered merge policy change, but I might be wrong. haven't ran gitbisect.
```
TestIndexWriterMergePolicy > testStressUpdateSameDocumentWithMergeOnCommit FAILED
java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.apache.lucene.util.ArrayUtil.growExact(ArrayUtil.java:320)
at org.apache.lucene.util.ArrayUtil.growInRange(ArrayUtil.java:346)
at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
at org.apache.lucene.util.BytesRefArray.append(BytesRefArray.java:67)
at org.apache.lucene.index.FieldUpdatesBuffer.append(FieldUpdatesBuffer.java:204)
at org.apache.lucene.index.FieldUpdatesBuffer.addUpdate(FieldUpdatesBuffer.java:173)
at org.apache.lucene.index.BufferedUpdates.addNumericUpdate(BufferedUpdates.java:139)
at org.apache.lucene.index.DocumentsWriterDeleteQueue$DocValuesUpdatesNode.apply(DocumentsWriterDeleteQueue.java:504)
at org.apache.lucene.index.DocumentsWriterDeleteQueue$DeleteSlice.apply(DocumentsWriterDeleteQueue.java:364)
at org.apache.lucene.index.DocumentsWriterPerThread.prepareFlush(DocumentsWriterPerThread.java:398)
at org.apache.lucene.index.DocumentsWriter.lambda$doFlush$9(DocumentsWriter.java:497)
at org.apache.lucene.index.DocumentsWriter$$Lambda/0x000003fc011b3758.get(Unknown Source)
at org.apache.lucene.index.DocumentsWriterFlushQueue.addTicket(DocumentsWriterFlushQueue.java:43)
at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:496)
at org.apache.lucene.index.DocumentsWriter.maybeFlush(DocumentsWriter.java:456)
at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:649)
at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3677)
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:4125)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:4087)
at org.apache.lucene.tests.index.RandomIndexWriter.commit(RandomIndexWriter.java:414)
at org.apache.lucene.tests.index.RandomIndexWriter.commit(RandomIndexWriter.java:370)
at org.apache.lucene.index.TestIndexWriterMergePolicy.stressUpdateSameDocumentWithMergeOnX(TestIndexWriterMergePolicy.java:808)
at org.apache.lucene.index.TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnCommit(TestIndexWriterMergePolicy.java:747)
at java.base/java.lang.invoke.LambdaForm$DMH/0x000003fc01128000.invokeVirtual(LambdaForm$DMH)
at java.base/java.lang.invoke.LambdaForm$MH/0x000003fc010a8800.invoke(LambdaForm$MH)
at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:153)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20, name=Thread-3, state=RUNNABLE, group=TGRP-TestIndexWriterMergePolicy]
Caused by:
java.lang.OutOfMemoryError: GC overhead limit exceeded
at __randomizedtesting.SeedInfo.seed([C13C284249671712]:0)
at org.apache.lucene.util.ArrayUtil.growExact(ArrayUtil.java:320)
at org.apache.lucene.util.ArrayUtil.growInRange(ArrayUtil.java:346)
at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
at org.apache.lucene.index.FieldUpdatesBuffer.add(FieldUpdatesBuffer.java:144)
at org.apache.lucene.index.FieldUpdatesBuffer.addUpdate(FieldUpdatesBuffer.java:175)
at org.apache.lucene.index.BufferedUpdates.addNumericUpdate(BufferedUpdates.java:139)
at org.apache.lucene.index.DocumentsWriterDeleteQueue$DocValuesUpdatesNode.apply(DocumentsWriterDeleteQueue.java:504)
at org.apache.lucene.index.DocumentsWriterDeleteQueue$DeleteSlice.apply(DocumentsWriterDeleteQueue.java:364)
at org.apache.lucene.index.DocumentsWriterPerThread.prepareFlush(DocumentsWriterPerThread.java:398)
at org.apache.lucene.index.DocumentsWriter.lambda$doFlush$9(DocumentsWriter.java:497)
at org.apache.lucene.index.DocumentsWriter$$Lambda/0x000003fc011b3758.get(Unknown Source)
at org.apache.lucene.index.DocumentsWriterFlushQueue.addTicket(DocumentsWriterFlushQueue.java:43)
at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:496)
at org.apache.lucene.index.DocumentsWriter.maybeFlush(DocumentsWriter.java:456)
at org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:404)
at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:447)
at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1552)
at org.apache.lucene.index.IndexWriter.softUpdateDocument(IndexWriter.java:1870)
at org.apache.lucene.tests.index.RandomIndexWriter.updateDocument(RandomIndexWriter.java:319)
at org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:781)
at org.apache.lucene.index.TestIndexWriterMergePolicy$$Lambda/0x000003fc011e5438.run(Unknown Source)
at java.base/java.lang.Thread.runWith(Thread.java:1583)
at java.base/java.lang.Thread.run(Thread.java:1570)
java.lang.AssertionError: org.apache.lucene.store.AlreadyClosedException: This DocumentsWriterDeleteQueue is already closed
at __randomizedtesting.SeedInfo.seed([C13C284249671712]:0)
at org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:787)
at java.base/java.lang.Thread.run(Thread.java:1570)
Caused by:
org.apache.lucene.store.AlreadyClosedException: This DocumentsWriterDeleteQueue is already closed
at app//org.apache.lucene.index.DocumentsWriterDeleteQueue.ensureOpen(DocumentsWriterDeleteQueue.java:308)
at app//org.apache.lucene.index.DocumentsWriterDeleteQueue.freezeGlobalBuffer(DocumentsWriterDeleteQueue.java:220)
at app//org.apache.lucene.index.DocumentsWriterPerThread.prepareFlush(DocumentsWriterPerThread.java:393)
at app//org.apache.lucene.index.DocumentsWriter.lambda$doFlush$9(DocumentsWriter.java:497)
at app//org.apache.lucene.index.DocumentsWriterFlushQueue.addTicket(DocumentsWriterFlushQueue.java:43)
at app//org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:496)
at app//org.apache.lucene.index.DocumentsWriter.maybeFlush(DocumentsWriter.java:456)
at app//org.apache.lucene.index.DocumentsWriter.preUpdate(DocumentsWriter.java:391)
at app//org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:413)
at app//org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1552)
at app//org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1837)
at app//org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1477)
at app//org.apache.lucene.tests.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:224)
at app//org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:783)
... 1 more
java.lang.AssertionError: org.apache.lucene.store.AlreadyClosedException: flush control is closed
at __randomizedtesting.SeedInfo.seed([C13C284249671712]:0)
at org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:787)
at java.base/java.lang.Thread.run(Thread.java:1570)
Caused by:
org.apache.lucene.store.AlreadyClosedException: flush control is closed
at app//org.apache.lucene.index.DocumentsWriterFlushControl.obtainAndLock(DocumentsWriterFlushControl.java:552)
at app//org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:415)
at app//org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1552)
at app//org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1837)
at app//org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1477)
at app//org.apache.lucene.tests.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:224)
at app//org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:783)
... 1 more
java.lang.AssertionError: pendingNumDocs 60573 != 60476 totalMaxDoc
at __randomizedtesting.SeedInfo.seed([C13C284249671712]:0)
at org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2463)
at org.apache.lucene.index.IndexWriter.maybeCloseOnTragicEvent(IndexWriter.java:5765)
at org.apache.lucene.index.IndexWriter.tragicEvent(IndexWriter.java:5755)
at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1556)
at org.apache.lucene.index.IndexWriter.softUpdateDocuments(IndexWriter.java:1597)
at org.apache.lucene.tests.index.RandomIndexWriter.updateDocument(RandomIndexWriter.java:315)
at org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:781)
at java.base/java.lang.Thread.run(Thread.java:1570)
```
### Gradle command to reproduce
```
./gradlew test --tests TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnCommit -Dtests.seed=C13C284249671712 -Dtests.nightly=true -Dtests.locale=rwk-TZ -Dtests.timezone=Antarctica/Macquarie -Dtests.asserts=true -Dtests.file.encoding=UTF-8
```
Contributor guide
Research direction
Run the supplied Gradle command with the listed seed and nightly settings, then read TestIndexWriterMergePolicy.java around testStressUpdateSameDocumentWithMergeOnCommit and stressUpdateSameDocumentWithMergeOnX. Trace the reported merge, flush, and FieldUpdatesBuffer frames to identify the cause; done means the nightly test completes without the OOM or follow-on failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, search, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100