quickwit-oss / quickwit-oss/quickwit
Rebuilding the known split ids set ended up not halving its size
Open
Nobody has claimed this yet.
bug
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
I got this warning message after apply #3891:
quickwit_indexing::actors::merge_planner: Rebuilding the known split ids set ended up not halving its size. Please report. This is likely a bug, please report. known_split_ids_len_after=9211 known_split_ids_len_before=17237
The problem is that the known_split_ids did not remove splits except on rebuilding.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the quickwit_indexing::actors::merge_planner entry point and trace how known_split_ids is maintained after apply #3891. Check the rebuilding warning and the before/after counts in the issue. Done means obsolete split IDs are removed during normal processing and the warning no longer occurs for this case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100