Should we have an interface VectorValues which would be implemented by [Byte/Float]VectorValues classes
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
Currently, there is lot of code duplication due to [ByteVectorValues](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/index/ByteVectorValues.java) and [FloatVectorValues](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/index/FloatVectorValues.java) (Example - [one](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java#L48-L73), [two](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java#L109-L301) etc.) because they are handled separately as one returns byte[] and other float[] for `vectorValue` method. The idea is if we should have a common interface `VectorValues`? that provides the common functionality and we could eventually get rid of this duplication. As a first step maybe we could try to move the below methods(common in above 2 classes) into its different interface `VectorValues`?
- int dimension()
- int size()
- byte[]/float[] vectorValue()
There could be some other ideas as well(maybe better) to address the code duplication here :
1. Making `DISI` implementing an interface which `VectorValues`(interface in [this PR](https://github.com/apache/lucene/pull/12636)) would extend making further room to remove duplication (or)
2. Making a wrapper over `ByteVectorValues` and `FloatVectorValues` etc?.
Looking for suggestions or more ideas on how to better handle this and avoid this current duplication.
Contributor guide
Research direction
Start by comparing ByteVectorValues.java and FloatVectorValues.java, then inspect the duplicated paths in KnnVectorsWriter.java and the VectorValues interface proposed in pull request 12636. Review the listed common methods and the alternative designs in the issue; done would require an agreed approach that reduces the duplication without losing the byte[] and float[] vector behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100