typesense / typesense/typesense

Union search hangs indefinitely when prefix query token contains 2+ multibyte UTF-8 characters

Open
#2,995 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
26.6k
Forks
973
Avg merge
18h 45m
Merged PRs (30d)
4

Description

Description

A multi_search request with union: true hangs forever (never returns a response) when the query token:

  1. contains two or more multibyte (non-ASCII) UTF-8 characters, and
  2. is a prefix of an indexed token (prefix search enabled, which is the default), and
  3. union: true is set.

The exact same query returns instantly as a regular (non-union) multi_search, and also returns instantly with prefix=false. Tokens with only one multibyte character work fine. The server stays healthy while the request hangs — only the affected request spins (looks like an infinite loop in the union prefix-expansion path).

We hit this in production with Danish food searches, e.g. a user typing blåbæ on the way to blåbær (blueberry).

Steps to reproduce

Fresh single-node Docker container, no other data:

docker run -d -p 8108:8108 typesense/typesense:31.0.rc10 --data-dir /tmp --api-key=test123

curl -s "http://localhost:8108/collections" -X POST \
  -H "X-TYPESENSE-API-KEY: test123" -H "Content-Type: application/json" \
  -d '{"name": "repro", "fields": [{"name": "name", "type": "string"}]}'

curl -s "http://localhost:8108/collections/repro/documents" -X POST \
  -H "X-TYPESENSE-API-KEY: test123" -H "Content-Type: application/json" \
  -d '{"name": "blåbær"}'

# This request never returns:
curl -s "http://localhost:8108/multi_search?page=1&per_page=10" -X POST \
  -H "X-TYPESENSE-API-KEY: test123" -H "Content-Type: application/json" \
  -d '{"union": true, "searches": [{"collection": "repro", "q": "blåbæ", "query_by": "name"}]}'

Expected behavior

The union search returns the prefix match blåbær, like the non-union search does.

Actual behavior

The request hangs indefinitely (we observed >60s in production until our proxy killed the connection; >15s locally before aborting client-side). /health keeps returning {"ok": true} the whole time.

Characterization matrix

Same collection as above (docs: blåbær, blåbærsyltetøj, jordbær, røræg, chicken), all with union: true unless noted:

Query Non-ASCII chars in token Prefix-matches indexed token Result
blåbæ 2 (å, æ) yes (blåbær) hangs forever
røræ 2 (ø, æ) yes (røræg) hangs forever
jordbæ 1 (æ) yes (jordbær) 200 OK
blåb 1 (å) yes 200 OK
blåbær (complete word) 2 n/a 200 OK
chicke 0 yes (chicken) 200 OK
rødgrø 3 no match 200 OK
blåbæ with prefix=false 2 200 OK
blåbæ non-union multi_search 2 yes 200 OK

Multi-token queries where only the last token is the prefix (e.g. blåbær med flø, 1 multibyte char in last token) also work.

Environment

  • Typesense 31.0.rc10 (also reproduced on 31.0.rc3)
  • Official Docker image, single node, default settings
  • Reproduced both on a clean local container and on our production cluster

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the hang using the Docker command and curl request in the issue, then trace the union=true multi_search prefix-expansion path for UTF-8 tokens. No source file or test is named; done means the blåbæ prefix returns blåbær promptly under union search, while the listed non-union, prefix=false, and single-character cases remain working.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, docker
Domain
backend, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.