typesense / typesense/typesense
Union search hangs indefinitely when prefix query token contains 2+ multibyte UTF-8 characters
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 26.6k
- Forks
- 973
- Avg merge
- 18h 45m
- Merged PRs (30d)
- 4
Description
Description
A multi_search request with union: true hangs forever (never returns a response) when the query token:
- contains two or more multibyte (non-ASCII) UTF-8 characters, and
- is a prefix of an indexed token (prefix search enabled, which is the default), and
union: trueis set.
The exact same query returns instantly as a regular (non-union) multi_search, and also returns instantly with prefix=false. Tokens with only one multibyte character work fine. The server stays healthy while the request hangs — only the affected request spins (looks like an infinite loop in the union prefix-expansion path).
We hit this in production with Danish food searches, e.g. a user typing blåbæ on the way to blåbær (blueberry).
Steps to reproduce
Fresh single-node Docker container, no other data:
docker run -d -p 8108:8108 typesense/typesense:31.0.rc10 --data-dir /tmp --api-key=test123
curl -s "http://localhost:8108/collections" -X POST \
-H "X-TYPESENSE-API-KEY: test123" -H "Content-Type: application/json" \
-d '{"name": "repro", "fields": [{"name": "name", "type": "string"}]}'
curl -s "http://localhost:8108/collections/repro/documents" -X POST \
-H "X-TYPESENSE-API-KEY: test123" -H "Content-Type: application/json" \
-d '{"name": "blåbær"}'
# This request never returns:
curl -s "http://localhost:8108/multi_search?page=1&per_page=10" -X POST \
-H "X-TYPESENSE-API-KEY: test123" -H "Content-Type: application/json" \
-d '{"union": true, "searches": [{"collection": "repro", "q": "blåbæ", "query_by": "name"}]}'
Expected behavior
The union search returns the prefix match blåbær, like the non-union search does.
Actual behavior
The request hangs indefinitely (we observed >60s in production until our proxy killed the connection; >15s locally before aborting client-side). /health keeps returning {"ok": true} the whole time.
Characterization matrix
Same collection as above (docs: blåbær, blåbærsyltetøj, jordbær, røræg, chicken), all with union: true unless noted:
| Query | Non-ASCII chars in token | Prefix-matches indexed token | Result |
|---|---|---|---|
blåbæ |
2 (å, æ) | yes (blåbær) |
hangs forever |
røræ |
2 (ø, æ) | yes (røræg) |
hangs forever |
jordbæ |
1 (æ) | yes (jordbær) |
200 OK |
blåb |
1 (å) | yes | 200 OK |
blåbær (complete word) |
2 | n/a | 200 OK |
chicke |
0 | yes (chicken) |
200 OK |
rødgrø |
3 | no match | 200 OK |
blåbæ with prefix=false |
2 | — | 200 OK |
blåbæ non-union multi_search |
2 | yes | 200 OK |
Multi-token queries where only the last token is the prefix (e.g. blåbær med flø, 1 multibyte char in last token) also work.
Environment
- Typesense
31.0.rc10(also reproduced on31.0.rc3) - Official Docker image, single node, default settings
- Reproduced both on a clean local container and on our production cluster
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the hang using the Docker command and curl request in the issue, then trace the union=true multi_search prefix-expansion path for UTF-8 tokens. No source file or test is named; done means the blåbæ prefix returns blåbær promptly under union search, while the listed non-union, prefix=false, and single-character cases remain working.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, docker
- Domain
- backend, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100