apache / apache/lucene

[Ideation] Improve readInts24 & readDelta16 performance for DocIdsWriter

Open
#12,917 0 comments 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

While working on [enhancing the performance of range queries](https://github.com/opensearch-project/OpenSearch/issues/11251), I noticed majority CPU cycles being spent on [readInts24()](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/util/bkd/DocIdsWriter.java#L326) & [readDelta16()](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/util/bkd/DocIdsWriter.java#L319).

I noticed a [PR](https://github.com/apache/lucene/issues/12527) trying to optimize readInts24() by reducing the number of sys calls, but the results did not resulted in considerable improvements.

Trying to gather inputs from community on other areas of improvements to be explored in the 2 methods listed above.

Contributor guide

Open the contributing guide

Research direction

Start by reading the readInts24() and readDelta16() implementations in lucene/core/src/main/java/org/apache/lucene/util/bkd/DocIdsWriter.java, then review the linked discussion about readInts24() optimization. Profile or benchmark both methods to identify a worthwhile improvement; done requires a concrete optimization direction with measurable performance gains.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.