quickwit-oss / quickwit-oss/quickwit
Ingestion failure when recreating an index
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Describe the bug
When re-creating an index, sometimes the control plane schedules the old pipeline and indexing fails. It might be because the reschedule takes too long to happen or another bug.
Steps to reproduce (if applicable)
Integration test ingest_tests::test_ingest_recreated_index
Expected behavior
A clear and concise description of what you expected to happen.
Configuration:
Please provide:
- Output of
quickwit --version - The index_config.yaml
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with quickwit/quickwit-integration-tests/src/tests/ingest_tests.rs, especially ingest_tests::test_ingest_recreated_index, and run the integration test to reproduce the failure. Trace how index recreation schedules pipelines in the control plane, focusing on whether the old pipeline can be scheduled. Done means the recreated-index ingestion test passes reliably without the old pipeline causing indexing failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100