quickwit-oss / quickwit-oss/quickwit
Aggregation failing after index schema update
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Describe the bug
We have had an index on which we defined a text field change with fast: false , this index already has data. We could not perform aggregation queries on it. When we attempted aggregation queries on it we got this error:
{ ERROR quickwit_search::error: search internal error: search failed for the following splits: x, y, z, . For instance, split x failed with the following error message: internal error: `An invalid argument was passed: 'Field \"change\" is not configured as fast field'`"}
We have now updated the schema to set fast: true so we could perform aggregation even if old split will be considered as missing data. however we still the same error after changing the schema and ingesting new data.
{ ERROR quickwit_search::error: search internal error: search failed for the following splits: x, y, z, . For instance, split x failed with the following error message: internal error: `An invalid argument was passed: 'Field \"change\" is not configured as fast field'`"}
Steps to reproduce (if applicable)
Steps to reproduce the behavior:
- create an index with a field
testthat's is not fast-field - ingest some data
- change the schema to make the field
testfast-fields - ingest more data
- attempt to make aggregation query on the field
test - You should be able to observe the reported error
Expected behaviour
We expected after the schema update, that aggregation queries on change field should work although it will not take into account old split on which fast-field was not enabled on the change field.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue by creating an index with a non-fast test field, ingesting data, enabling fast fields, ingesting more data, and running an aggregation. Trace schema updates, ingestion, and aggregation over old and new splits; done means the aggregation succeeds while treating old splits as missing data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100