quickwit-oss / quickwit-oss/quickwit
Ingester behavior when disk is full
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Currently, ingester may end up accepting persist request when their disk is full.
If the OS buffer is not full, no error might be returned.
We need to poll-check for the disk usage, and change the behavior of quickwit when it goes above
a threshold.
The behavior is yet to be decided. Probably, the closest thing is decommissionning: close all shards and not accept the creation of new shards. In addition, it might not be possible to run indexing/merge pipelines; which could really make the control plane's task hard.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named. Start by tracing ingester persist requests and the shard-creation, indexing, and merge-pipeline paths, then review how decommissioning is handled. Done requires an agreed disk-usage threshold and defined behavior for existing shards, new shard creation, and affected pipelines, with coverage for full-disk conditions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, distributed-systems, infrastructure
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100