Lucene 9.11: Make AbstractKnnVectorQuery.searchLeaf() overridable to enable custom KNN instrumentation
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
### Problem
In Lucene 9.11, the `searchLeaf()` method in `AbstractKnnVectorQuery` is declared `private`, preventing custom subclasses from overriding the search execution logic. This restricts the ability to build custom instrumentation or extend search behaviors.
Our team at Uber is working on improving visibility and control over KNN query execution in a fork of Lucene 9.11. We developed functionality to track exact vs. approximate search usage and fallback patterns, which is currently impossible to get from Lucene itself.
To enable this, we created a subclass of `KnnFloatVectorQuery` that overrides `searchLeaf()` to route execution through custom logic. However, due to the method’s `private` access modifier in 9.11, this override is not possible without modifying the core Lucene code. While we currently use a custom jar with the modified access for testing, we cannot safely ship this setup to production or rely on it long-term.
### Solution
Change the access modifier of `searchLeaf()` method from private to public in Lucene 9.11, as already implemented in [commit d279af1](https://github.com/apache/lucene/commit/d279af1961f40465a3a9e8ab84848afe69fd8424) on the main branch. This change is already upstream and therefore changing it with this PR presents minimal logic changes and maintains full backward compatibility with all existing usages.
Contributor guide
Research direction
Start by locating AbstractKnnVectorQuery.searchLeaf() and the KnnFloatVectorQuery subclass mentioned in the issue, then compare the access modifier with commit d279af1 on the main branch. The work is complete when custom subclasses can override searchLeaf() without modifying a local Lucene jar and the relevant Java checks pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, search
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100