quickwit-oss / quickwit-oss/quickwit

Elasticsearch API: a term query on a string field matches only the exact value, and no `.keyword` subfield exists

Open
#6,787 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
11.7k
Forks
597
Avg merge
2d 22h
Merged PRs (30d)
37

Description

Repository: quickwit-oss/quickwit
Version: 0.8.2 (quickwit/quickwit:0.8.2)
Reproduced first-hand: yes, on 2026-09-08, against Elasticsearch 8.15.0 as the reference.

What happens

With a dynamically mapped index, a term query for error on a string field:

returns
Elasticsearch 8.15.0, term on level [lower, mixed, norank]
Quickwit 0.8.2, term on level [lower, norank]
Elasticsearch 8.15.0, term on level.keyword [lower, norank]
Quickwit 0.8.2, term on level.keyword []

mixed is the document whose value is "Error Occurred". Elasticsearch's
dynamic mapping makes a string field text — analyzed, so error matches one
of its tokens — with a keyword subfield that is not analyzed. Quickwit's
dynamic mapping declares tokenizer: "raw", so the field is exact-only and no
.keyword subfield is created.

Both halves change results silently. A query written against Elasticsearch
returns fewer rows on the first, and nothing at all on the second — and
field.keyword is the idiom every Elasticsearch dashboard uses for exact
matching and aggregation.

Reproduction

docker run -d --name qw -p 7280:7280 quickwit/quickwit:0.8.2 run
until curl -sf localhost:7280/health/livez >/dev/null; do sleep 1; done

curl -s -X POST localhost:7280/api/v1/indexes -H 'Content-Type: application/json' \
  -d '{"version":"0.8","index_id":"textdemo","doc_mapping":{"mode":"dynamic"}}'

printf '%s\n' \
 '{"index":{"_index":"textdemo"}}' '{"doc":"lower","level":"error"}' \
 '{"index":{"_index":"textdemo"}}' '{"doc":"mixed","level":"Error Occurred"}' \
 '{"index":{"_index":"textdemo"}}' '{"doc":"norank","level":"error"}' \
 | curl -s -X POST localhost:7280/api/v1/_elastic/textdemo/_bulk \
     -H 'Content-Type: application/x-ndjson' --data-binary @-

# A newly created index defaults to commit_timeout_secs: 60.
sleep 70

for f in level level.keyword; do
  curl -s -X POST localhost:7280/api/v1/_elastic/textdemo/_search \
    -H 'Content-Type: application/json' \
    -d "{\"query\":{\"term\":{\"$f\":\"error\"}}}" \
    | python3 -c 'import json,sys; print([h["_source"]["doc"] for h in json.load(sys.stdin)["hits"]["hits"]])'
done

Is this a bug or a mapping choice?

Genuinely unsure, which is why this is a question as much as a report. A
different tokenizer can be configured, and Quickwit's dynamic default is a
reasonable one for a store that is not trying to be Elasticsearch everywhere.

But the default is what a user gets, and the endpoint is
/api/v1/_elastic/, which sets the expectation that Elasticsearch queries
work. If matching Elasticsearch's dynamic mapping is out of scope, saying so in
the Elasticsearch-compatibility documentation would be enough — the surprise is
what costs people time, not the choice.

Where this came from

SpecMatrix, a conformance corpus for observability backends. The check is
cases/es-bulk/text-vs-keyword-term.yaml, whose expectation was taken by
running the query against Elasticsearch and copying what it returned.


Found by SpecMatrix, a conformance corpus for observability backends. Happy to be told this is configuration or already known — the check will record whichever it turns out to be.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the /api/v1/_elastic/ endpoint and the dynamic mapping behavior described in the reproduction. Review cases/es-bulk/text-vs-keyword-term.yaml and compare its expectations with Quickwit's compatibility behavior; done means the intended term-query behavior or the Elasticsearch-compatibility limitation is clearly established and covered by documentation or a regression check.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, search
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.