quickwit-oss / quickwit-oss/quickwit
Add a not-so-happy path for the deletion of shards in ingesters upon source/index deletion
Open
Nobody has claimed this yet.
bug
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Follow up on #4244
If the attempt to remove the shard in the ingester fails (ingerster temporarily unavailable for isntance)
we need a way to reliably make sure they eventually get the message.
Above 1mn or so, the lack of truncate message for the shards will block mrecordlog.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing #4244 and tracing the shard-removal flow triggered by source or index deletion. Investigate how an unavailable ingester can eventually receive the removal or truncate message, and verify that mrecordlog is no longer blocked after the retry path completes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100