quickwit-oss / quickwit-oss/quickwit

Slow query performance when using sort_by_field on queries with few matches

Open
#6,440 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Rust
Stars
11.7k
Forks
597
Avg merge
2d 22h
Merged PRs (30d)
37

Description

Describe the bug
I noticed that the following query becomes significantly slower when sort_by_field is specified:

When I remove sort_by_field, the same query becomes very fast. I also observed a large increase in fast_field_cache evict_bytes while running the query.

However, the query only returns fewer than 10 documents, which is far below max_hits. Based on this behavior, it seems that the sort operation may be scanning or loading fast field data before applying the query filter. If that is the case, it appears inefficient and unexpected.

{"query":"trace_id:\"0002e641f5b84161f35b907c8fce3039\"","max_hits":100,"start_timestamp":1771303321,"end_timestamp":1779079322,"sort_by_field":"span_start_timestamp_nanos"}

Steps to reproduce (if applicable)

  1. Ingest a large number of traces so that the fast-field cache capacity is exceeded.
  2. Run a trace query with sort_by_field enabled.
  3. Compare the latency and fast_field_cache evict_bytes with the same query executed without sort_by_field.

Expected behavior
The query filter should be applied first, and sorting should only be performed on the matched documents. In this case, since the query matches fewer documents, adding sort_by_field should not significantly increase query latency or cause substantial fast-field cache eviction.

Configuration:

  1. v0.9.0-rc
  2. Index template: official otel-traces-v0_9 index

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the trace-query path that handles sort_by_field and the fast_field_cache, using the supplied query as a reproduction. Compare execution and cache behavior with and without sorting under a large index, then confirm that a query matching fewer than 10 documents no longer causes substantial latency or cache eviction.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
performance, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.