apache / apache/lucene

Strange ConcurrentMergeScheduler behavior with intra-merge threads

Open
#13,478 4 comments 0 reactions 0 assignees View on GitHub
affects-version:9.11.0 type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

It was noticed that the CMS intra-merge behavior was not fully tested. In an effort to do this, a change to override when the intra-merge scheduler is used has been drafted. https://github.com/apache/lucene/pull/13475

This PR has exposed many existing assertions that may all be weird testing failures. But a couple are more worrisome.

In particular:

```
./gradlew test --tests TestCodecHoldsOpenFiles.test -Dtests.seed=AA3C2FCDA0773874 -Dtests.locale=ar-PS -Dtests.timezone=Asia/Magadan -Dtests.asserts=true -Dtests.file.encoding=UTF-8
```

Will periodically fail with an NPE on the `sort`:

```
org.apache.lucene.index.MergePolicy$MergeException: java.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.Sort.getSort()" because the return value of "org.apache.lucene.index.LeafMetaData.getSort()" is null
at __randomizedtesting.SeedInfo.seed([AA3C2FCDA0773874]:0)
at app//org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:764)
at app//org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:756)

Caused by:
java.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.Sort.getSort()" because the return value of "org.apache.lucene.index.LeafMetaData.getSort()" is null
at org.apache.lucene.index.SortingCodecReader.assertCreatedOnlyOnce(SortingCodecReader.java:739)
at org.apache.lucene.index.SortingCodecReader.getOrCreate(SortingCodecReader.java:721)
at org.apache.lucene.index.SortingCodecReader.getOrCreateDV(SortingCodecReader.java:759)
at org.apache.lucene.index.SortingCodecReader$7.getNumeric(SortingCodecReader.java:571)
at org.apache.lucene.codecs.DocValuesConsumer$1.getNumeric(DocValuesConsumer.java:201)
at org.apache.lucene.codecs.lucene90.Lucene90DocValuesConsumer$1.getSortedNumeric(Lucene90DocValuesConsumer.java:138)
at org.apache.lucene.codecs.lucene90.Lucene90DocValuesConsumer.writeValues(Lucene90DocValuesConsumer.java:188)
at org.apache.lucene.codecs.lucene90.Lucene90DocValuesConsumer.addNumericField(Lucene90DocValuesConsumer.java:133)
at org.apache.lucene.tests.codecs.asserting.AssertingDocValuesFormat$AssertingDocValuesConsumer.addNumericField(AssertingDocValuesFormat.java:87)
at org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:183)
at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:142)
at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:151)
at org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:205)
at org.apache.lucene.index.SegmentMerger.mergeWithLogging(SegmentMerger.java:325)
at org.apache.lucene.index.SegmentMerger.lambda$merge$1(SegmentMerger.java:155)
at org.apache.lucene.search.TaskExecutor$TaskGroup.lambda$createTask$0(TaskExecutor.java:117)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at org.apache.lucene.index.ConcurrentMergeScheduler$CachedExecutor.lambda$execute$0(ConcurrentMergeScheduler.java:982)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
```

### Version and environment details

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with PR 13475 and reproduce TestCodecHoldsOpenFiles.test using the provided Gradle command and seed. Trace the ConcurrentMergeScheduler and SortingCodecReader stack shown in the failure, then verify that the intermittent null sort failure no longer occurs and the test passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.