apache / apache/lucene

Lucene 9.11: Make AbstractKnnVectorQuery.searchLeaf() overridable to enable custom KNN instrumentation

Open
#15,020 6 comments 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

### Problem
In Lucene 9.11, the `searchLeaf()` method in `AbstractKnnVectorQuery` is declared `private`, preventing custom subclasses from overriding the search execution logic. This restricts the ability to build custom instrumentation or extend search behaviors.
Our team at Uber is working on improving visibility and control over KNN query execution in a fork of Lucene 9.11. We developed functionality to track exact vs. approximate search usage and fallback patterns, which is currently impossible to get from Lucene itself.
To enable this, we created a subclass of `KnnFloatVectorQuery` that overrides `searchLeaf()` to route execution through custom logic. However, due to the method’s `private` access modifier in 9.11, this override is not possible without modifying the core Lucene code. While we currently use a custom jar with the modified access for testing, we cannot safely ship this setup to production or rely on it long-term.

### Solution
Change the access modifier of `searchLeaf()` method from private to public in Lucene 9.11, as already implemented in [commit d279af1](https://github.com/apache/lucene/commit/d279af1961f40465a3a9e8ab84848afe69fd8424) on the main branch. This change is already upstream and therefore changing it with this PR presents minimal logic changes and maintains full backward compatibility with all existing usages.

Contributor guide

Open the contributing guide

Research direction

Start by locating AbstractKnnVectorQuery.searchLeaf() and the KnnFloatVectorQuery subclass mentioned in the issue, then compare the access modifier with commit d279af1 on the main branch. The work is complete when custom subclasses can override searchLeaf() without modifying a local Lucene jar and the relevant Java checks pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, search
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.