quickwit-oss / quickwit-oss/quickwit
Optimize the Quickwit collector
Open
@PSeitz is already working on this.
Since Oct 18, 2023.
enhancement
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Following #3978.
There are a lot of headroom in our collector. Some of them are explored in #3915.
- implement collect_batch (measured 30% improvement)
- plug the count optimization when possible
- rely on the new quickwit TopK collector
- remove the SortOrder from the sorting key
- in presence of several sorting key, lazily access the second one (is it common though?)
- filter on value pre-monotonic mapping
- etc.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.