typesense / typesense/typesense

Quoted search terms ignore exact phrase matching when combined with other quoted phrases

Open
#2,050 2 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
26.6k
Forks
973
Avg merge
18h 45m
Merged PRs (30d)
4

Description

Description

When performing searches with quoted phrases, Typesense appears to ignore the exact matching requirement when multiple quoted strings are present in the query.

Steps to reproduce

  1. Run Typesense locally
docker run --rm -p 8108:8108 typesense/typesense:27.1 --api-key=abcdef --data-dir=/home
  1. Create collection
curl -X POST 'http://localhost:8108/collections' \
-H 'X-Typesense-Api-Key: abcdef' \
-H 'Content-Type: application/json' \
-d '{
  "name": "titles",
  "fields": [
    {"name": "title", "type": "string", "index": true}
  ]
}'
  1. Insert test data
curl -X POST 'http://localhost:8108/collections/titles/documents/import' \
-H 'X-Typesense-Api-Key: abcdef' \
-H 'Content-Type: application/jsonl' \
-d '{"title": "full stack developer"}
    {"title": "full stack javascript developer"}'
  1. Searching "full stack" foobar properly returns 0 results:
  curl --request GET \
  --url 'http://localhost:8108/collections/titles/documents/search?q=%22full%20%20stack%22%20foobar&query_by=title' \
  --header 'X-TYPESENSE-API-KEY: abcdef'

Output:

{
  "facet_counts": [],
  "found": 0,
  "hits": [],
  "out_of": 2,
  "page": 1,
  "request_params": {
    "collection_name": "titles",
    "first_q": "\"full  stack\" foobar",
    "per_page": 10,
    "q": "\"full  stack\" foobar"
  },
  "search_cutoff": false,
  "search_time_ms": 0
}
  1. Adding quotes around "foobar" should still return 0 results but returns 2 results:
curl --request GET \
  --url 'http://localhost:8108/collections/titles/documents/search?q=%22full%20%20stack%22%20%22foobar%22&query_by=title' \
  --header 'X-TYPESENSE-API-KEY: abcdef'

Output:

{
  "facet_counts": [],
  "found": 2,
  "hits": [
    {
      "document": { "id": "1", "title": "full stack javascript developer" },
      "highlight": {
        "title": {
          "matched_tokens": ["full", "stack"],
          "snippet": "<mark>full</mark> <mark>stack</mark> javascript developer"
        }
      },
      "highlights": [
        {
          "field": "title",
          "matched_tokens": ["full", "stack"],
          "snippet": "<mark>full</mark> <mark>stack</mark> javascript developer"
        }
      ],
      "text_match": 100015,
      "text_match_info": {
        "best_field_score": "48",
        "best_field_weight": 213,
        "fields_matched": 7,
        "num_tokens_dropped": 1,
        "score": "100015",
        "tokens_matched": 0,
        "typo_prefix_score": 255
      }
    },
    {
      "document": { "id": "0", "title": "full stack developer" },
      "highlight": {
        "title": {
          "matched_tokens": ["full", "stack"],
          "snippet": "<mark>full</mark> <mark>stack</mark> developer"
        }
      },
      "highlights": [
        {
          "field": "title",
          "matched_tokens": ["full", "stack"],
          "snippet": "<mark>full</mark> <mark>stack</mark> developer"
        }
      ],
      "text_match": 100015,
      "text_match_info": {
        "best_field_score": "48",
        "best_field_weight": 213,
        "fields_matched": 7,
        "num_tokens_dropped": 1,
        "score": "100015",
        "tokens_matched": 0,
        "typo_prefix_score": 255
      }
    }
  ],
  "out_of": 2,
  "page": 1,
  "request_params": {
    "collection_name": "titles",
    "first_q": "\"full  stack\" \"foobar\"",
    "per_page": 10,
    "q": "\"full  stack\" \"foobar\""
  },
  "search_cutoff": false,
  "search_time_ms": 0
}
  1. Two sets of quoted strings, "full stack" "javascript developer", properly returns only 1 result:
curl --request GET \
  --url 'http://localhost:8108/collections/titles/documents/search?q=%22full%20stack%22%20%22javascript%20developer%22&query_by=title' \
  --header 'X-TYPESENSE-API-KEY: abcdef'

Output:

{
  "facet_counts": [],
  "found": 1,
  "hits": [
    {
      "document": { "id": "1", "title": "full stack javascript developer" },
      "highlight": {
        "title": {
          "matched_tokens": ["full", "stack", "javascript", "developer"],
          "snippet": "<mark>full</mark> <mark>stack</mark> <mark>javascript</mark> <mark>developer</mark>"
        }
      },
      "highlights": [
        {
          "field": "title",
          "matched_tokens": ["full", "stack", "javascript", "developer"],
          "snippet": "<mark>full</mark> <mark>stack</mark> <mark>javascript</mark> <mark>developer</mark>"
        }
      ],
      "text_match": 100015,
      "text_match_info": {
        "best_field_score": "48",
        "best_field_weight": 213,
        "fields_matched": 7,
        "num_tokens_dropped": 1,
        "score": "100015",
        "tokens_matched": 0,
        "typo_prefix_score": 255
      }
    }
  ],
  "out_of": 2,
  "page": 1,
  "request_params": {
    "collection_name": "titles",
    "first_q": "\"full stack\" \"javascript developer\"",
    "per_page": 10,
    "q": "\"full stack\" \"javascript developer\""
  },
  "search_cutoff": false,
  "search_time_ms": 0
}
  1. Adding a word not in the data set in the second quoted string, "full stack" "javascript foobar developer", wrongly returns 2 results:
curl --request GET \
  --url 'http://localhost:8108/collections/titles/documents/search?q=%22full%20stack%22%20%22javascript%20foobar%20developer%22&query_by=title' \
  --header 'X-TYPESENSE-API-KEY: abcdef'

Output:

{
  "facet_counts": [],
  "found": 2,
  "hits": [
    {
      "document": { "id": "1", "title": "full stack javascript developer" },
      "highlight": {
        "title": {
          "matched_tokens": ["full", "stack", "javascript", "developer"],
          "snippet": "<mark>full</mark> <mark>stack</mark> <mark>javascript</mark> <mark>developer</mark>"
        }
      },
      "highlights": [
        {
          "field": "title",
          "matched_tokens": ["full", "stack", "javascript", "developer"],
          "snippet": "<mark>full</mark> <mark>stack</mark> <mark>javascript</mark> <mark>developer</mark>"
        }
      ],
      "text_match": 100015,
      "text_match_info": {
        "best_field_score": "48",
        "best_field_weight": 213,
        "fields_matched": 7,
        "num_tokens_dropped": 1,
        "score": "100015",
        "tokens_matched": 0,
        "typo_prefix_score": 255
      }
    },
    {
      "document": { "id": "0", "title": "full stack developer" },
      "highlight": {
        "title": {
          "matched_tokens": ["full", "stack", "developer"],
          "snippet": "<mark>full</mark> <mark>stack</mark> <mark>developer</mark>"
        }
      },
      "highlights": [
        {
          "field": "title",
          "matched_tokens": ["full", "stack", "developer"],
          "snippet": "<mark>full</mark> <mark>stack</mark> <mark>developer</mark>"
        }
      ],
      "text_match": 100015,
      "text_match_info": {
        "best_field_score": "48",
        "best_field_weight": 213,
        "fields_matched": 7,
        "num_tokens_dropped": 1,
        "score": "100015",
        "tokens_matched": 0,
        "typo_prefix_score": 255
      }
    }
  ],
  "out_of": 2,
  "page": 1,
  "request_params": {
    "collection_name": "titles",
    "first_q": "\"full stack\" \"javascript foobar developer\"",
    "per_page": 10,
    "q": "\"full stack\" \"javascript foobar developer\""
  },
  "search_cutoff": false,
  "search_time_ms": 1
}

Expected Behavior

  1. Quoted strings should be matched exactly as whole phrases
  2. Multiple quoted strings in a query should all need to match for a document to be returned

Actual Behavior

Adding a search term not found in the dataset into a second quoted string in the query will return results as if the second quoted string was not part of the query.

Metadata

Typesense Version: 27.1

OS: Docker

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the Docker Typesense 27.1 setup and the collection/search requests shown in the report. Trace how the documents search endpoint handles multiple quoted phrases, then verify that every quoted phrase is matched exactly and that queries containing an absent term return no results.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.