apache / apache/lucene

Take advantage of bloom filter when delete terms

Open
#12,725 8 comments 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Today we delete terms in a [seeking forward fashion with `seekCeil`](https://github.com/apache/lucene/blob/09da2291c501e25c38a2cb69899b69afb65a22e5/lucene/core/src/java/org/apache/lucene/index/FrozenBufferedUpdates.java#L602). This can help short cut terms seek by comparing to the next existed term. But the `seekCeil` can not take advantage of [`bloom filter`](https://github.com/apache/lucene/blob/09da2291c501e25c38a2cb69899b69afb65a22e5/lucene/codecs/src/java/org/apache/lucene/codecs/bloom/BloomFilteringPostingsFormat.java#L331) which also significantly helped primary key lookups. I wonder if it is a acceptable trade off to use `seekExact` here for potential bloom filter implementation, or we can have a way to take advantage of both short-cuts (E.g. add `mightContains` for `TermsEnum`) ?

(I'd think this is not only the problem for delete terms but all scenes that need to seek sorted terms)

Contributor guide

Open the contributing guide

Research direction

Start with FrozenBufferedUpdates.java around the seekCeil-based delete-term logic and BloomFilteringPostingsFormat.java around the bloom-filter lookup described in the issue. Compare seekExact, seekCeil, and the proposed TermsEnum mightContains approach, then determine the needed design and validation for preserving both shortcuts; done requires an agreed implementation direction and corresponding behavior checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.