apache / apache/lucene

Concurrent byte and int block implementations [LUCENE-2575]

Open
#3,649 51 comments 0 reactions 0 assignees View on GitHub
affects-version:Realtime Branch legacy-jira-fix-version:6.0 legacy-jira-priority:Major module:core/index type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

The current \*BlockPool implementations aren't quite concurrent.
We really need something that has a locking flush method, where
flush is called at the end of adding a document. Once flushed,
the newly written data would be available to all other reading
threads (ie, postings etc). I'm not sure I understand the slices
concept, it seems like it'd be easier to implement a seekable
random access file like API. One'd seek to a given position,
then read or write from there. The underlying management of byte
arrays could then be hidden?

---
Migrated from [LUCENE-2575](https://issues.apache.org/jira/browse/LUCENE-2575) by Jason Rutherglen, updated May 09 2016
Attachments: [LUCENE-2575.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2575/LUCENE-2575.patch) (versions: 8)
Linked issues:
- #3641
- #3388

Contributor guide

Open the contributing guide

Research direction

Start by reading the issue body, the attached LUCENE-2575.patch, and linked issues #3641 and #3388 to understand the existing BlockPool and slices design. Done means concurrent byte and int block implementations provide a locking flush at document completion and make newly flushed data visible to reader threads.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.