apache / apache/lucene

A Mirroring Split Policy For Load-Balancing Search Requests [LUCENE-2434]

Open
#3,508 0 comments 0 reactions 0 assignees View on GitHub
affects-version:3.0.1 legacy-jira-priority:Major module:core/index type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

(Please note that the feature described in #3499 is required reading for this issue.)

The mirroring split policy builds on the remoting split policy, treating each sub-index as a mirror of the super-index (upon which the split writer is built). The motivation for this policy is to allow a given index to be available on multiple nodes (that define a cluster), which helps to load-balance search requests across the nodes in that cluster, not to mention implement fail-over for index searchers.

Unlike other mirroring solutions for Lucene, this approach gives users the ability to (a) pick and choose the types of directories underlying the mirror, and (b) add/remove mirrors on the fly. However, note that the latter capability has not yet been implemented, although it will be soon.

---
Migrated from [LUCENE-2434](https://issues.apache.org/jira/browse/LUCENE-2434) by Karthick Sankarachary
Attachments: [LUCENE-2434.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2434/LUCENE-2434.patch)
Linked issues:
- #3499

Contributor guide

Open the contributing guide

Research direction

Read linked issue #3499 first, then inspect the attached LUCENE-2434.patch to understand the proposed split-policy design. Confirm the intended scope for mirroring sub-indexes, selectable directory types, load-balanced search, failover, and the explicitly unfinished add/remove-mirrors capability before defining completion.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.