apache / apache/lucene

Add Shape Support to BKD (extend to an R*/X-Tree data structure) [LUCENE-7110]

Open
#8,165 10 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

I've been tinkering with this off and on for a while and its showing some promise so I'm going to open an issue to (eventually) add this feature to a future release.

R\*/X-Tree is a data structure designed to support Shapes (2D, 3D, nD) where, like the internal node, the key for each leaf node is the Minimum Bounding Range (MBR - sometimes "incorrectly" referred to as Minimum Bounding Rectangle) of the shape. Inserting a shape then boils down to the best way of optimizing the tree structure. This optimization is driven by a set of criteria for choosing the appropriate internal key (e.g., minimizing overlap between siblings, maximizing "squareness", minimizing area, maximizing space usage). Query is then (a bit oversimplified) a two-phase process:
- recurse each branch that overlaps with the MBR of the query shape
- compute the relation with the leaf node(s) - in higher dimensions (3+) this becomes an increasingly difficult computational geometry problem

The current BKD implementation is a special simplified case of an R\*/X tree where, for Point data, it is always guaranteed there will never be overlap between sibling nodes (because you're using the point data as the keys). By exploiting this property the tree algorithms (split, merge, etc) are relatively cheap (hence their performance boost over postings based numerics). By modifying the key data, and extending the tree generation algorithms BKD logic can be extended to support Shape data using the MBR as the Key and modifying split and merge based on the criteria needed for optimizing a shape-based data structure.

The initial implementation (based on limitations of the GeoAPI) will support 2D shapes only. Once the GeoAPI can performantly handle 3D shapes the change is relatively trivial to add the third dimension to the tree generation code.

Like everything else, this feature will be created in sandbox and, once mature, will graduate (likely to lucene-spatial or lucene-spatial-extras depending on the library needs).

---
Migrated from [LUCENE-7110](https://issues.apache.org/jira/browse/LUCENE-7110) by Nick Knize (@nknize), updated Jan 23 2018

Contributor guide

Open the contributing guide

Research direction

Start by reading Lucene's current BKD implementation and the sandbox work mentioned in the issue, then review the stated GeoAPI limitations for 2D shapes. Determine the required tree-generation, split, merge, and query changes for shape data before attempting implementation; done means a mature sandbox feature that supports the described 2D shape behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.