quickwit-oss / quickwit-oss/quickwit

Search requests with high `max_hits` (> ~100) causes fetch error with indexes stored on Cloudflare R2.

Open
#1,777 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Rust
Stars
11.7k
Forks
597
Avg merge
2d 22h
Merged PRs (30d)
37

Description

Describe the bug
Search requests with high max_hits (> ~100) causes fetch errors with indexes stored on Cloudflare R2.

This is likely because Quickwit goes over the Cloudflare R2 limit of 1000 GetObject requests per second: https://developers.cloudflare.com/r2/platform/limits/#account-plan-limits

As discussed on discord, it would be helpful if Quickwit propagates the Cloudflare R2 error message to confirm this theory.

Steps to reproduce (if applicable)
Steps to reproduce the behavior:

  1. Search request:
❯ curl http://127.0.0.1:7280/api/v1/reddit_comments_v05/search\?query\=frankfurt\&max_hits\=1000\&sort_by_field\=-created_utc
{
  "InternalError": "Internal error: `Error when fetching docs for splits [\"01G7YXYNFFVCQBY8Y2B5078EWW\", \"01G7YXPQQM8DER0F8112WK5C2P\", \"01G7YY6GH3N259P137D7TF6PA7\", \"01G7YWQHPA2Q1PHK8B0BXS0ZJP\"]: searcher-doc-async\n\nCaused by:\n    An IO error occurred: 'Failed to fetch slice 1448721986..1449072261 for object: s3://test-bucket/indexes/reddit_comments_v05/01G7YXPQQM8DER0F8112WK5C2P.split'.`."
}%

Configuration:
Please provide:

  1. Output of quickwit --version
❯ RUST_LOG=quickwit=debug bin/quickwit --version
Quickwit 0.3.1nightly

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the curl search against indexes stored on Cloudflare R2 with a high max_hits value, using the configuration and Quickwit version shown in the issue. Trace the searcher's fetch path from the reported “Failed to fetch slice” error and ensure the underlying Cloudflare R2 message is propagated; done means the response identifies the storage error rather than only reporting an internal fetch failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.