quickwit-oss / quickwit-oss/quickwit
Quickwit's indexing per-index footprint
Open
@fulmicoton is already working on this.
Since Jun 7, 2022.
biggy
enhancement
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Right now Quickwit uses several thread PER indexing pipeline.
This ticket is about improving the situation. In the following N is hte number of indexes.
- Avoid having N x k concurrent uploads by sharing the conccurrent upload semaphore,
- Have configurable runtime
- Add a no more message handler in the actor framwork
- Modify the indexer actor to make sure we have at most K indexwriter at the same time.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.