quickwit-oss / quickwit-oss/quickwit
Cache TermInfo and Postinglist/Positions
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
We can expect some common terms to be searched for repeatedly.
Currently we execute 3 requests.
- Fetch TermInfo
- Fetch posting list
- (Optional) Fetch position list
TermInfo is very small and should provide excellent cost/benefit ratio in a LRUHashmap<Term, TermInfo>, potentially saving one get request.
Posting and position lists are more complicated on a good cost/benefit hashing strategy, because they can become large and vary in size.
Ideally the costs of a term would be reflected on its data access pattern. That means very large lists need to be retrieved more often than smaller lists (Data access linear to its size).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by tracing the search request path that fetches TermInfo, posting lists, and optional position lists, then inspect existing cache abstractions. Done means defining and validating an LRU cache strategy whose access cost reflects the size and retrieval patterns of these lists.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100