quickwit-oss / quickwit-oss/quickwit
Reorder split search to use CPU as soon as possible
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Some split may have their data in the fast field cache for instance.
We want these splits to run first, to start "consuming" CPU as soon as possible.
We could also consider sparkling a few split of smaller size in the beginnig.
Or maybe shuffle the order of splits.
The large splits go first strategy might not be optimal here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by locating split search scheduling and the current large-splits-first ordering, then inspect how fast field cache availability and split size affect execution; done requires a decided ordering strategy with validation that CPU starts consuming work sooner.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100