quickwit-oss / quickwit-oss/quickwit
Sort splits per checkpoint in the merge policy
Open
Nobody has claimed this yet.
backlog
enhancement
low-priority
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Right we sort splits per timestamp.
It might be more interesting to sort them per checkpoint (which is a bit ill defined) and save that metadata in the metastore.
That way each split would be (and stay) a clear section of the original message queue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the merge policy and metastore components to understand how splits are currently ordered and what checkpoint metadata exists. Clarify the checkpoint definition and verify that split ordering and persisted metadata keep each split as a stable section of the original message queue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100