quickwit-oss / quickwit-oss/quickwit

Improve user XP when using delete API

Open
#2,494 1 comment 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
11.7k
Forks
597
Avg merge
2d 22h
Merged PRs (30d)
37

Description

When a user post a valid delete query, he will receive the following response OK

{
  "create_timestamp": 1000000,
  "opstamp": 3,
  "delete_query": {
     "index_id": "my-index",
     "start_timestamp": 1000000,
     "query": "body:trash"
  }
}

This is fine but... the user will probably want to check if the documents to be deleted are still there or not. He will typically do a search query to check that. Unfortunately, he may still find some documents even if there are no more deletes running...
This is explained by the fact that some documents can be in immature splits and they will be deleted later when the split becomes mature.
But the user has no way to check that or to check that the delete query has finished. The consequence is that he will believe there is a bug in Quickwit.

We can improve in different ways this XP. For example:

  • add new search parameters to filter out splits on maturity and delete_opstamp. This will enable us to return accurate results.
  • add a query params that will return all splits that contains matching documents.
  • add an endpoint on the delete API that will check which splits need to undergo some delete operation
  • and possibly more.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the delete API response and the search behavior described in the issue, paying attention to immature splits and delete_opstamp. Compare the proposed search parameters, matching-splits query, and delete-status endpoint; done means users can determine whether deletes have finished and obtain accurate search results.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.