apache / apache/lucene

ArrayIndexOutOfBoundsException during indexing [LUCENE-10441]

Open
#11,477 2 comments 0 reactions 0 assignees View on GitHub
affects-version:8.10 legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Hi experts!, i have facing ArrayIndexOutOfBoundsException during indexing and committing documents, this exception gives me no clue about what happened so i have little information for debugging, can i have some suggest about what could be and how to fix this error? i'm using Lucene 8.10.0

```java
java.lang.ArrayIndexOutOfBoundsException: -1
    at org.apache.lucene.util.BytesRefHash$1.get(BytesRefHash.java:179)
    at org.apache.lucene.util.StringMSBRadixSorter$1.get(StringMSBRadixSorter.java:42)
    at org.apache.lucene.util.StringMSBRadixSorter$1.setPivot(StringMSBRadixSorter.java:63)
    at org.apache.lucene.util.Sorter.binarySort(Sorter.java:192)
    at org.apache.lucene.util.Sorter.binarySort(Sorter.java:187)
    at org.apache.lucene.util.IntroSorter.quicksort(IntroSorter.java:41)
    at org.apache.lucene.util.IntroSorter.quicksort(IntroSorter.java:83)
    at org.apache.lucene.util.IntroSorter.sort(IntroSorter.java:36)
    at org.apache.lucene.util.MSBRadixSorter.introSort(MSBRadixSorter.java:133)
    at org.apache.lucene.util.MSBRadixSorter.sort(MSBRadixSorter.java:126)
    at org.apache.lucene.util.MSBRadixSorter.sort(MSBRadixSorter.java:121)
    at org.apache.lucene.util.BytesRefHash.sort(BytesRefHash.java:183)
    at org.apache.lucene.index.SortedSetDocValuesWriter.flush(SortedSetDocValuesWriter.java:171)
    at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:348)
    at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:228)
    at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
    at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:476)
    at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:656)
    at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3364)
    at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3770)
    at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3728)
```

---
Migrated from [LUCENE-10441](https://issues.apache.org/jira/browse/LUCENE-10441) by Peixin Li, updated Mar 14 2022
Linked issues:
- #9660

Contributor guide

Open the contributing guide

Research direction

Start with the stack-trace entry points in src/java/org/apache/lucene/util/BytesRefHash.java and src/java/org/apache/lucene/index/SortedSetDocValuesWriter.java. Reproduce the failure using the reported Lucene 8.10.0 indexing and commit flow, then compare the behavior with linked issue #9660. Done means identifying the triggering input or confirming a reproducible Lucene defect and documenting the cause and fix path.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.