apache / apache/lucene

Hypothetical perf improvements in DocValuesRangeQuery: reducing comparisons for some queries/segments [LUCENE-7618]

Open
#8,669 3 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

In reviewing the DocValuesRangeQuery code, it occured to me that there _might_ be some potential performance optimizations possible in a few cases relating queries that involve explicitly specified open ranges (ie: min or max are null) or in the case of SortedSet: range queries that are **effectively** open ended on particular segments, because the min/max are below/above the minOrd/maxOrd for the segment.

Since these seemed like semi-common situations (open ended range queries are fairly common in my experience, i'm not sure about the secondary SortedSet "ord" case, but it seemd potentially promising particularly for fields like incrementing ids, or timestamps, where values are added sequentially and likeley to be clustered together) I did a bit of experimenting and wanted to post my findings in jira – patch & details to follow in comments.

---
Migrated from [LUCENE-7618](https://issues.apache.org/jira/browse/LUCENE-7618) by Chris M. Hostetter (@hossman), updated Jan 04 2017
Attachments: [LUCENE-7618.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-7618/LUCENE-7618.patch)

Contributor guide

Open the contributing guide

Research direction

Start by reading DocValuesRangeQuery and the attached LUCENE-7618.patch, then review the issue's experimental findings about open ranges and segment bounds. Done means determining whether the proposed comparison reductions are valid and beneficial for the described query cases, with supporting details or measurements.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.