apache / apache/lucene

EnumFieldSource throws AssertionError [LUCENE-9343]

Open
#10,383 0 comments 0 reactions 0 assignees View on GitHub
affects-version:8.5.1 legacy-jira-priority:Major module:core/query/scoring type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

When a doc id is smaller than a previsou one, I obtain an AssertionError. I check the Lucene code, and find that it is thrown from the following code of EnumFieldSource:

```java
public FunctionValues getValues(Map context, LeafReaderContext readerContext) throws IOException {
final NumericDocValues arr = DocValues.getNumeric(readerContext.reader(), field); return new IntDocValues(this) {
final MutableValueInt val = new MutableValueInt(); int lastDocID; private int getValueForDoc(int doc) throws IOException {
if (doc < lastDocID) {
throw new AssertionError("docs were sent out-of-order: lastDocID=" + lastDocID + " vs doc=" + doc);
}
lastDocID = doc;
int curDocID = arr.docID();
if (doc > curDocID) {
curDocID = arr.advance(doc);
}
if (doc == curDocID) {
return (int) arr.longValue();
} else {
return 0;
}
}
}
```

My code does not catch the exception. It catches IllegalArgumentException, because Lucene throws the exceptions for similar problems:

DocTermsIndexDocValues:

```java
protected int getOrdForDoc(int doc) throws IOException {
if (doc < lastDocID) {
throw new IllegalArgumentException("docs were sent out-of-order: lastDocID=" + lastDocID + " vs docID=" + doc);
}
lastDocID = doc;
int curDocID = termsIndex.docID();
if (doc > curDocID) {
curDocID = termsIndex.advance(doc);
}
if (doc == curDocID) {
return termsIndex.ordValue();
} else {
return -1;
}
}
```

BytesRefFieldSource:

```java
private BytesRef getValueForDoc(int doc) throws IOException {
if (doc < lastDocID) {
throw new IllegalArgumentException("docs were sent out-of-order: lastDocID=" + lastDocID + " vs docID=" + doc);
}
lastDocID = doc;
int curDocID = binaryValues.docID();
if (doc > curDocID) {
curDocID = binaryValues.advance(doc);
}
if (doc == curDocID) {
return binaryValues.binaryValue();
} else {
return null;
}
}
```

FloatFieldSource:

```java
private float getValueForDoc(int doc) throws IOException {
if (doc < lastDocID) {
throw new IllegalArgumentException("docs were sent out-of-order: lastDocID=" + lastDocID + " vs docID=" + doc);
}
lastDocID = doc;
int curDocID = arr.docID();
if (doc > curDocID) {
curDocID = arr.advance(doc);
}
if (doc == curDocID) {
return Float.intBitsToFloat((int)arr.longValue());
} else {
return 0f;
}
}
```

 EnumFieldSource throws a wrong exception. Can this problem be fixed?

---
Migrated from [LUCENE-9343](https://issues.apache.org/jira/browse/LUCENE-9343) by Ackel Filia

Contributor guide

Open the contributing guide

Research direction

Locate EnumFieldSource.getValues and compare its out-of-order handling with DocTermsIndexDocValues, BytesRefFieldSource, and FloatFieldSource. Reproduce the decreasing-doc-ID case described in the issue and verify that the resulting exception matches the behavior of the surrounding field sources.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.