Separate index/store document APIs take 2?
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
As @rmuir managed to make me look into reducing the amount of guessing we're doing in our document API, I think that a requirement for doing it right will be to split our index and store document APIs. Currently, `Document` and `IndexableField` are trying to cover both, which creates a confusing API.
For the store API, I wonder if we still need a Document abstraction. Maybe we could return a simple `List` and push the work on users to convert it into a map if they want to be able to access fields by name. This is something that is easy to do with Java streams nowadays.
For the index API, I'd like to model `IndexableField` according to how it's getting consumed by `IndexingChain`. Something like:
```java
class IndexableField {
// Populates the inverted index, IndexableValue wraps one of a BytesRef or a TokenStream
IndexableValue invertedValue(Analyzer analyzer, TokenStream reuse);
// Populates points
byte[] pointValue();
// Populates NUMERIC or SORTED_NUMERIC doc values
long numericDocValue();
// Populates BINARY, SORTED or SORTED_SET doc values
BytesRef docValue();
// Populates stored fields
StoredValue storedValue();
// Populates byte[] vectors
byte[] binaryVectorValue();
// Populates float[] vectors
float[] floatVectorValue();
}
```
I haven't thought much about it but I'm curious to hear thoughts.
Contributor guide
Research direction
Start by reading the current Document and IndexableField APIs alongside their use in IndexingChain. Review the proposed split between index and store APIs and the listed value methods. Done would require an agreed API design and defined changes, which this issue does not yet specify.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend-api-design, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100