TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnGetReader failure due to DWPT race condition
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
This replicates rarely for me locally. To me this indicates a race condition instead of some logic error.
```
java.lang.AssertionError: seqNo=114262 vs maxSeqNo=114261
at __randomizedtesting.SeedInfo.seed([905AD19F00667602]:0)
at org.apache.lucene.index.DocumentsWriterDeleteQueue.getNextSequenceNumber(DocumentsWriterDeleteQueue.java:567)
at org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:178)
at org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:155)
at org.apache.lucene.index.DocumentsWriterPerThread.finishDocuments(DocumentsWriterPerThread.java:328)
at org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:273)
at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:425)
at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1558)
at org.apache.lucene.index.IndexWriter.softUpdateDocuments(IndexWriter.java:1603)
at org.apache.lucene.tests.index.RandomIndexWriter.updateDocument(RandomIndexWriter.java:315)
at org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:781)
at java.base/java.lang.Thread.run(Thread.java:1583)
```
This is very similar to: https://github.com/elastic/elasticsearch/issues/105122#issuecomment-2098412230
Either we are updating `maxSeqNo` incorrectly, or there is a race condition in DWPT that both tests have hit
### Gradle command to reproduce
```
./gradlew test --tests TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnGetReader -Dtests.seed=905AD19F00667602 -Dtests.locale=chr-Cher-US -Dtests.timezone=MST7MDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
```
Contributor guide
Research direction
Start with TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnGetReader and run the supplied Gradle command with its seed. Trace the reported sequence-number assertion through DocumentsWriterDeleteQueue, DocumentsWriterPerThread, DocumentsWriter, IndexWriter, and RandomIndexWriter. Done means the suspected DWPT race or maxSeqNo problem is understood and the reproducer no longer fails.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, search, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100