PagingFieldCollector fails when sorting by String null fields [LUCENE-4814]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
When performing a paginated search sorted by a STRING field, and documents without this field are present in the resultset, PagingFieldCollector stops collecting documents (or collects all documents, depending on sorting order) when one of the these documents are used as "after" element for the pagination.
The reason i have observed is that:
org.apache.lucene.search.FieldComparator.TermOrdValComparator.compareDocToValue(int doc, BytesRef value) (line 1160) always returns 1 in that case because value is null ("after" document sorting field value) whereas docValue (returned by termsIndex.getTerm) is a 0-length BytesRef for documents without that field.
A simple patch can be performed changing only this method with no (apparently) side-effects:
Current:
if (docValue == null) {
if (value == null) {
Patched:
if (docValue == null || docValue.length==0) {
if (value == null || value.length==0) {
Please excuse my lack of rigour, I'll try to attach the patch.
Regards
---
Migrated from [LUCENE-4814](https://issues.apache.org/jira/browse/LUCENE-4814) by Ignacio del Valle Alles
Contributor guide
Research direction
Start at org.apache.lucene.search.FieldComparator.TermOrdValComparator.compareDocToValue(int doc, BytesRef value), around line 1160, and reproduce pagination with STRING-sorted results containing documents without that field. Verify the behavior for both sort directions and add or update regression coverage so paging continues correctly when a missing-field document is used as the after element.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100