sourcefuse / sourcefuse/loopback4-microservice-catalog
[search-service] Exact full name search fails when text contains special characters due to adjacency operator
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 297
- Forks
- 78
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 4
Description
Describe the bug
The PsqlQueryBuilder._formatAndSanitize() method strips all special characters from search queries and joins tokens with the <-> (FOLLOWED
BY/adjacency) operator. This causes searches to fail when the indexed text contains special characters (parentheses, hyphens, brackets, etc.)
because:
- Special characters are stripped from the search query but affect tokenization in the indexed
tsvector - The
<->operator requires tokens to be immediately adjacent, which fails when the original text has punctuation between words
Code Location: services/search-service/src/classes/psql/query.builder.ts (lines 80-87)
_formatAndSanitize(param: string) {
return param
.replace(/[^A-Za-z\s0-9]/g, ' ') // Strips ALL special characters
.split(' ')
.filter(p => p)
.map(p => `${p}:*`)
.join('<->'); // Adjacency operator - too strict
}
To Reproduce
Steps to reproduce the behavior:
- Create a searchable entity with a name containing special characters, e.g.,
"RCP Project (Phase 1)"or"ABC-123 Test Project" - Use the search endpoint to search for the exact full name
- Observe that no results are returned
Example:
- Entity name:
"Project ABC-123 (Test)" - Search query:
"Project ABC-123 (Test)" - Sanitized tsquery:
Project:* <-> ABC:* <-> 123:* <-> Test:* - Result: No match because
<->requires immediate token adjacency, but thetsvectorhas different positions due to punctuation
Expected behavior
Searching for an exact or partial name should return matching results regardless of special characters in the text. The search should be more
lenient with token proximity.
Proposed Solution
Replace the strict <-> (FOLLOWED BY) operator with one of:
-
`&` (AND) operator - Tokens must all exist but in any order/position
.join(' & '); // Less strict, allows any position -
`` (WITHIN N WORDS) operator - Allow some distance between tokens
.join('<2>'); // Tokens within 2 words of each other -
Configurable operator - Let users choose the matching strategy
.join(this.options.tokenJoinOperator ?? '<->');
Additional context
- Package version:
@sourceloop/search-service@9.0.0(also affects latest11.0.0) - Database: PostgreSQL with full-text search
- Impact: Users cannot find entities by their exact names when names contain common characters like hyphens, parentheses, brackets, slashes,
etc. - Frequency: Always reproducible (5/5)
- Environments affected: All environments using PostgreSQL full-text search
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in services/search-service/src/classes/psql/query.builder.ts, especially PsqlQueryBuilder._formatAndSanitize() and the search endpoint behavior described in the reproduction steps. Compare the adjacency, AND, and WITHIN operators against PostgreSQL full-text results for names containing hyphens and parentheses, then verify that exact and partial searches return matching entities without breaking existing matching behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, typescript
- Domain
- backend, databases, search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100