quickwit-oss / quickwit-oss/quickwit

Query can not filter

Open
#6,211 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Rust
Stars
11.7k
Forks
597
Avg merge
2d 22h
Merged PRs (30d)
37

Description

Describe the bug

Data exists, becase I already check with srcip:10.111.1.111 dstport:161 this filter. After I was curious, I just wanted the filter by eventtime.

srcip:10.111.1.111 dstport:161 eventtime:1773940811365322000

And the response is always empty.

My data is here,

{
  "eventtime": 1773940811365322000,
  "srcip": "10.111.1.111",
  "dstport":161,
  "type": "traffic",
}

Also I check with these queries.

  • srcip:10.111.1.111 dstport:161 eventtime:1773940811365322000*
  • srcip:10.111.1.111 dstport:161 eventtime:*1773940811365322000*
  • srcip:10.111.1.111 dstport:161 eventtime:*1773940811365322000*
  • srcip:10.111.1.111 dstport:161 eventtime:*17739408113*
  • srcip:10.111.1.111 dstport:161 eventtime:"*1*"

Configuration:
Quickwit 0.8.2 (x86_64-unknown-linux-gnu 2024-09-03T11:26:51Z 0f28194)

Indexing Settings

{
  "commit_timeout_secs": 30,
  "docstore_compression_level": 10,
  "docstore_blocksize": 2000000,
  "split_num_docs_target": 50000000,
  "merge_policy": {
    "type": "stable_log",
    "min_level_num_docs": 500000,
    "merge_factor": 10,
    "max_merge_factor": 12,
    "maturation_period": "2days"
  },
  "resources": {
    "heap_size": "6.0 GB"
  }
}

Doc Mapping

{
  "field_mappings": [
    {
      "name": "_time",
      "type": "datetime",
      "fast": true,
      "fast_precision": "seconds",
      "indexed": true,
      "input_formats": [
        "unix_timestamp"
      ],
      "output_format": "rfc3339",
      "stored": true
    },
    {
      "name": "logsource",
      "type": "text",
      "fast": false,
      "fieldnorms": false,
      "indexed": true,
      "record": "basic",
      "stored": true,
      "tokenizer": "raw"
    },
    {
      "name": "_msg",
      "type": "text",
      "fast": false,
      "fieldnorms": false,
      "indexed": true,
      "record": "basic",
      "stored": true,
      "tokenizer": "default"
    }
  ],
  "tag_fields": [],
  "store_source": false,
  "index_field_presence": false,
  "timestamp_field": null,
  "mode": "dynamic",
  "dynamic_mapping": {
    "indexed": true,
    "tokenizer": "raw",
    "record": "basic",
    "stored": true,
    "expand_dots": true,
    "fast": {
      "normalizer": "raw"
    }
  },
  "max_num_partitions": 200,
  "tokenizers": []
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported eventtime queries against Quickwit 0.8.2 using the supplied document, indexing settings, and dynamic mapping. No source file or test is named, so trace the query filtering and dynamic-field handling entry points; done means the exact eventtime filter returns the matching document or the supported behavior is clarified with a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.