quickwit-oss / quickwit-oss/quickwit
Updating doc mapping does not update index V2 validation
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Describe the bug
We can update a doc mapping with PUT api/v1/indexes/<index id>, indexing pipelines are automatically restarted, but when we hit the ingest V2 endpoint after the update, documents obeying the new doc mapping are likely be considered as invalid by existing shards that still use the previous doc mapping.
Steps to reproduce (if applicable)
Test was added in https://github.com/quickwit-oss/quickwit/pull/5739/files
Expected behavior
The best solution is likely to force the recreation of all shards for the existing index
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the PUT api/v1/indexes/ update flow and the ingest V2 validation path, then review the reproduction test added in pull request 5739. Trace how existing shards retain the previous doc mapping after an update. Done means documents matching the new mapping are accepted by all shards after the mapping changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100