apache / apache/lucene

Change spatial RPT default prefixGridScanLevel [LUCENE-6340]

Open
#7,401 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major module:spatial type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

The "prefixGridScanLevel" tuning knob of RPT was shown to work pretty well based on benchmarking many years ago against point data. It defaults to 4 up from the max levels in the tree. However for non-point data, I've seen a benchmark where it's default setting hurt a ton. Setting it to a large number like 100 (anything >= maxLevels) effectively disables it. We should pick the default based on the "pointsOnly" property of RPT – if you set pointsOnly you get the current default (scan the bottom 4 levels) otherwise you only scan the bottom level.

In addition, it's worth reviewing what gains there are for point-only data. Some testing of FlexPrefixTree last summer of point data showed it didn't seem to have much affect. But I'd like to investigate more rigorously as that wasn't primarily what we were looking for.

---
Migrated from [LUCENE-6340](https://issues.apache.org/jira/browse/LUCENE-6340) by David Smiley (@dsmiley)

Contributor guide

Open the contributing guide

Research direction

Start by locating RPT's prefixGridScanLevel and pointsOnly settings, then compare their defaults for point and non-point data. Done means pointsOnly retains the current bottom-four-level scan while other data scans only the bottom level, with additional benchmark results documenting whether the point-only setting provides meaningful gains.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
40/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.