apache / apache/lucene

Faster Query-Time Join [LUCENE-5291]

Open
#6,355 2 comments 0 reactions 0 assignees View on GitHub
affects-version:4.5 legacy-jira-label:join legacy-jira-label:query legacy-jira-priority:Minor module:core/index module:core/search type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

The current implementation of query-time join could be complemented with a much faster one, provided some choices can be made about what to join on.

Since join is really a database concept, we found it quite natural to restrict the keys to be integers and be single valued.

We found that if it is possible to use integers keys, and having single valued key fields, the speed of join can be improved 50 fold. Proper caching again speeds up about 20 times.

I'd like to contribute our code if you agree that it is a useful contribution. That probably depends on what you think of the choices we made about the keys, so that need to be discussed first?

---
Migrated from [LUCENE-5291](https://issues.apache.org/jira/browse/LUCENE-5291) by Erik Groeneveld, 3 votes, updated Oct 19 2013
Attachments: [LUCENE-5291.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5291/LUCENE-5291.patch)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the attached LUCENE-5291.patch and the issue's proposed query-time join constraints: integer, single-valued keys and caching. The issue does not identify source files or tests; done would require first resolving whether those design choices are acceptable and then defining and implementing the contribution.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.