apache / apache/lucene

Course cacheable spatial filters [LUCENE-5397]

Open
#6,460 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major module:spatial type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Sometimes, spatial filter queries aren't very cacheable because they are too precise (perhaps submitted from a user's mobile phone GPS coords). To make matters worse, many mobile user's move in-between one query and the next. What to do? You could simply not cache such queries because it's pointless, or perhaps we can optimize for this under-the-hood.

The idea here is to pre-compute the resulting set of very-course grid squares, and make this part of the cacheable filter query (e.g. the Query returned would probably be a MultiTermQuery or TermsFilter for a Filter). This component of the solution is designed to produce a filter that is likely a cache-hit, however it isn't precise enough. Then you could combine it with a precise spatial query that you don't cache (e.g. Solr LatLonType query).

Ultimately, the change here might simply be under-the-hood smarts in RecursivePrefixTreeStrategy that returns a query/filter as defined above when the precision given is clearly very course... but it is probably best to more explicitly say when this behavior is desired. And if we wanted to **really** optimize for this case, then a dedicated PrefixTree encoding could be developed that has overlapping grid squares.

(this is just a TODO idea; I'm not starting on this anytime soon)

---
Migrated from [LUCENE-5397](https://issues.apache.org/jira/browse/LUCENE-5397) by David Smiley (@dsmiley)

Contributor guide

Open the contributing guide

Research direction

Start by reading the issue's references to RecursivePrefixTreeStrategy, MultiTermQuery, TermsFilter, Solr LatLonType, and PrefixTree encoding. Determine whether the desired behavior should be an explicit opt-in or under-the-hood optimization for coarse spatial filters. Done would require an agreed design and an implementation that improves cache hits without losing precise spatial filtering.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.