pendingNumDocs doesn't match totalMaxDoc if tragedy on flush() [LUCENE-9621]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
While implementing a test to trigger an OutOfMemoryError on flush() in https://github.com/apache/lucene-solr/pull/2088, I noticed that the OOME was followed by an assertion failure on rollback with the following stacktrace:
```java
java.lang.AssertionError: pendingNumDocs 1 != 0 totalMaxDoc
at __randomizedtesting.SeedInfo.seed([ABBF17C4E0FCDEE5:DDC8E99910AFC8FF]:0)
at org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2398)
at org.apache.lucene.index.IndexWriter.maybeCloseOnTragicEvent(IndexWriter.java:5196)
at org.apache.lucene.index.IndexWriter.tragicEvent(IndexWriter.java:5186)
at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3932)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3874)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3853)
at org.apache.lucene.index.TestIndexWriterDelete.testDeleteAllRepeated(TestIndexWriterDelete.java:496)
```
We should probably look into how exactly we behave with this kind of tragedy on flush().
---
Migrated from [LUCENE-9621](https://issues.apache.org/jira/browse/LUCENE-9621) by Michael Froh (@msfroh), updated Dec 11 2020
Contributor guide
Research direction
Start with IndexWriter.doFlush(), rollbackInternal(), and maybeCloseOnTragicEvent(), then inspect TestIndexWriterDelete.testDeleteAllRepeated() and the reported stack trace. Reproduce the OutOfMemoryError during flush and verify that rollback completes without the pendingNumDocs/totalMaxDoc assertion failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100