apache / apache/lucene

PagingFieldCollector fails when sorting by String null fields [LUCENE-4814]

Open
#5,879 0 comments 0 reactions 0 assignees View on GitHub
affects-version:4.1 legacy-jira-label:null legacy-jira-label:pagination legacy-jira-label:sorting legacy-jira-priority:Minor module:core/search type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

When performing a paginated search sorted by a STRING field, and documents without this field are present in the resultset, PagingFieldCollector stops collecting documents (or collects all documents, depending on sorting order) when one of the these documents are used as "after" element for the pagination.

The reason i have observed is that:
org.apache.lucene.search.FieldComparator.TermOrdValComparator.compareDocToValue(int doc, BytesRef value) (line 1160) always returns 1 in that case because value is null ("after" document sorting field value) whereas docValue (returned by termsIndex.getTerm) is a 0-length BytesRef for documents without that field.

A simple patch can be performed changing only this method with no (apparently) side-effects:
Current:
if (docValue == null) {
if (value == null) {
Patched:
if (docValue == null || docValue.length==0) {
if (value == null || value.length==0) {

Please excuse my lack of rigour, I'll try to attach the patch.
Regards

---
Migrated from [LUCENE-4814](https://issues.apache.org/jira/browse/LUCENE-4814) by Ignacio del Valle Alles

Contributor guide

Open the contributing guide

Research direction

Start at org.apache.lucene.search.FieldComparator.TermOrdValComparator.compareDocToValue(int doc, BytesRef value), around line 1160, and reproduce pagination with STRING-sorted results containing documents without that field. Verify the behavior for both sort directions and add or update regression coverage so paging continues correctly when a missing-field document is used as the after element.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.