Text-index does not support multi-token substring search where first and last tokens are partial
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
We've seen many cases where a user wants to search for a substring in a field with a text index. If all of the tokens in the query are complete words, we can directly use a phrase search:
`SELECT * FROM table WHERE text_match("col", '"substring match query"')`
However, if the first or last token is a partial word (e.g., `"string match que"`), the query will not return any results. Treating the query as regex text-match query does not work either as Pinot only supports regex match on a single token.
To work around the limitation, we can use this a query like this:
`SELECT * FROM table WHERE text_match("col", '/*string/ AND match AND /que*/') AND "col" LIKE "%string match que%"`
But this is very slow and computationally expensive due to the LIKE which is necessary for validating the order of the tokens.
Contributor guide
Assessment
This issue has not been assessed yet.