quickwit-oss / quickwit-oss/quickwit
Remove the need for tantivy::Document and allocations
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
We've already experimented with borrowed formats (TantivyValue<'a>), but another approach could be for tantivy to drive a visitor that emits a stream of field values such as FieldValue<'a> { field: Cow<'a, [u8]>, value: Value<'a> }. That would remove most allocations.
Follow up for #2834
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the follow-up issue #2834 and the existing use of tantivy::Document, TantivyValue<'a>, and FieldValue<'a>. Compare the proposed visitor-driven stream with the current value flow. Done means tantivy no longer needs tantivy::Document and removes most allocations without losing field values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100