dotCMS / dotCMS/core

Content Search: escape Lucene query-syntax characters in content type field filters

Open
#37,532 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

OKR : Customer Support UPL
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Problem Statement

In the Content Search portlet, filtering on a content type field (e.g. Headline) returns "No results found" whenever the value contains characters that carry meaning in the Lucene query_string syntax — : ( ) / ". The user gets a silent empty result set for content they know exists, with no indication that anything failed.

Current behaviour

ContentletAjax.searchContentletsByUser(), text-field branch — ContentletAjax.java:874–906. The field value is split on [,|\s+], each token is wrapped in wildcards, and the clauses are concatenated as:

+(.:token ._dotraw:token)

No escaping is applied to the tokens. Tokens such as (XETRA: and NYSE: therefore produce an invalid query. The resulting exception is caught and swallowed at ContentletAjax.java:1058 (Logger.error(..., "Unable to execute Lucene Query", pe)) while totalHits stays 0, so the UI renders an empty result set rather than an error.

Why this is raised as an enhancement rather than a defect fix

Per discussion with engineering: the legacy field-level query generation is overly verbose and does not reliably produce correct queries beyond this specific case (required per-token wildcard clauses across both the analyzed field and _dotraw, no empty-token filtering — consecutive whitespace currently emits **). The preference is to rework the construction rather than patch the escaping in isolation. Content Drive is intended to replace the Content Search portlet, so the improved field-level search behaviour should land there.

Steps to Reproduce
  • Content → Search, select a content type with a text field.
  • In that field's filter, enter a value containing a colon, parenthesis or forward slash, e.g.
  • ABC Bank (XETRA: DBKGn.DB / NYSE: DB) and PSL Launch independent European CLO Total Return Indices
  • Result: Showing 0-0 of 0, "No results found".
  • Remove the special characters and repeat → the item is found.
Acceptance Criteria
  • Values entered into content type field filters are treated as literal text, not query syntax
  • Verified across the full reserved set: \ + - ! ( ) : ^ [ ] " { } ~ * ? | & /
  • Consecutive whitespace no longer produces empty ** clauses
  • A query that fails to execute surfaces an error state in the UI instead of a silent zero-result set
  • / added to the escape set used by the global catchall search path
  • Regression test covering the reported headline value
  • Equivalent field-filter behaviour confirmed in Content Drive
dotCMS Version

Evergreen

Severity

High - Major functionality broken

Links

https://helpdesk.dotcms.com/a/tickets/39185

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ContentletAjax.java:874–906 and the error handling at line 1058 to understand the current field-filter query and silent failure. Then trace the equivalent field-filter behavior in Content Drive. Done means literal handling for the reserved characters, no empty wildcard clauses, surfaced query errors, the global catchall slash fix, and regression coverage for the reported headline value.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.