Content Search: escape Lucene query-syntax characters in content type field filters
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 970
- Forks
- 486
- Avg merge
- 3d 33m
- Merged PRs (30d)
- 170
Description
Problem Statement
In the Content Search portlet, filtering on a content type field (e.g. Headline) returns "No results found" whenever the value contains characters that carry meaning in the Lucene query_string syntax — : ( ) / ". The user gets a silent empty result set for content they know exists, with no indication that anything failed.
Current behaviour
ContentletAjax.searchContentletsByUser(), text-field branch — ContentletAjax.java:874–906. The field value is split on [,|\s+], each token is wrapped in wildcards, and the clauses are concatenated as:
+(.:token ._dotraw:token)
No escaping is applied to the tokens. Tokens such as (XETRA: and NYSE: therefore produce an invalid query. The resulting exception is caught and swallowed at ContentletAjax.java:1058 (Logger.error(..., "Unable to execute Lucene Query", pe)) while totalHits stays 0, so the UI renders an empty result set rather than an error.
Why this is raised as an enhancement rather than a defect fix
Per discussion with engineering: the legacy field-level query generation is overly verbose and does not reliably produce correct queries beyond this specific case (required per-token wildcard clauses across both the analyzed field and _dotraw, no empty-token filtering — consecutive whitespace currently emits **). The preference is to rework the construction rather than patch the escaping in isolation. Content Drive is intended to replace the Content Search portlet, so the improved field-level search behaviour should land there.
Steps to Reproduce
- Content → Search, select a content type with a text field.
- In that field's filter, enter a value containing a colon, parenthesis or forward slash, e.g.
- ABC Bank (XETRA: DBKGn.DB / NYSE: DB) and PSL Launch independent European CLO Total Return Indices
- Result: Showing 0-0 of 0, "No results found".
- Remove the special characters and repeat → the item is found.
Acceptance Criteria
- Values entered into content type field filters are treated as literal text, not query syntax
- Verified across the full reserved set: \ + - ! ( ) : ^ [ ] " { } ~ * ? | & /
- Consecutive whitespace no longer produces empty ** clauses
- A query that fails to execute surfaces an error state in the UI instead of a silent zero-result set
- / added to the escape set used by the global catchall search path
- Regression test covering the reported headline value
- Equivalent field-filter behaviour confirmed in Content Drive
dotCMS Version
Evergreen
Severity
High - Major functionality broken
Links
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ContentletAjax.java:874–906 and the error handling at line 1058 to understand the current field-filter query and silent failure. Then trace the equivalent field-filter behavior in Content Drive. Done means literal handling for the reserved characters, no empty wildcard clauses, surfaced query errors, the global catchall slash fix, and regression coverage for the reported headline value.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100