dotCMS / dotCMS/core

Content Drive: Search & Performance at Scale

Open
#36,814 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dotCMS : Content Management Epic Team : Scout
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

What we need

Content Drive has to stay fast and return the right thing on instances with millions of assets. Today it does neither.

  • Search and pagination can take up to seven minutes on large repositories.
  • Search returns results that don't contain the search term — searching IMG for a file named IMG_1004.jpg surfaces unrelated files such as og-preview.vtl.

Both hit the everyday path, not an edge case: browsing a folder, moving to the next page, typing part of a filename you know exists.

Why it matters

Content Drive replaced Site Browser and Content Search with one workspace. On a demo instance it behaves well. On a real enterprise repository it doesn't, and at that point the unified workspace is worse than the tools it replaced — at least Content Search came back quickly.

Seven minutes is past the point where a user assumes the page has hung. And once a result set contains entries a user can't account for, the whole result set stops being trustworthy — so they search again with different terms, which multiplies the cost of the slowness.

Asset filenames matter here specifically. They're a primary way people locate things, especially developers looking for VTLs, CSS, and theme files.

What "done" looks like

On an instance with millions of assets:

  • A user types a fragment and gets matching results in a time that feels interactive.
  • The fragment is found whether it lives in the filename, title, metadata, path, or URL — all five.
  • Every returned result visibly relates to what was typed, and it's clear which field matched.
  • Paginating through a large folder is equally fast.

Acceptance criteria

  • Search matches filename, title, metadata, paths, and URLs
  • Results are ordered by a defined and documented precedence between those five — a filename match should not rank equally with a metadata match
  • The matched field is visible in the result row, so a path or metadata hit doesn't read as noise
  • Partial and prefix filename matching works, including underscores and extensions
  • A known filename is findable by typing a leading fragment of it
  • Every result matches the entered term in a field search is documented to cover — no unexplainable results
  • Search and pagination meet an agreed response-time target at millions-of-assets scale
  • Search behaves correctly when combined with the existing filter chips
  • Behaviour is verified at that scale, and protected against regression

Out of scope

  • The filter chip UX — owned by #35430 (Content Drive: Search). Same surface, so the two need coordinating.
  • New search capabilities — semantic or vector search, natural-language queries, AI-assisted search. This is about making literal search work; anything smarter builds on it.
  • New metadata or DAM features — auto-tagging, custom metadata schemas, where-used tracking.
  • Upload and workflow-action performance — unless they turn out to share a cause with this.

Notes

The seven-minute figure is a report, not a measurement. Worth confirming early, because it's also what sizes this work — at the moment we can't estimate it.

Whether the two problems share a root cause is unknown. If one thing is causing both the slowness and the bad relevance, this is much smaller than if they're independent. That answer changes the shape of the work, so it's worth knowing before committing to a plan.

Widening search to five surfaces makes the relevance bar harder, not easier. More legitimate matches means more results a user can't immediately explain, even when every one is technically correct. This is the part most likely to pass its tests and still feel wrong.

Tightening relevance will look like a regression to someone. Results people currently see will disappear. That's intended, but it needs saying out loud when it ships.

Open questions

  • What response-time target should we hold? Search-as-you-type and page-to-page navigation may warrant different numbers. Happy to set this once there's a measurement to set it against.
  • What precedence order across the five surfaces? A match in each is valid but they shouldn't rank equally. Product decision — flag it when you get there and we'll settle it.

References

  • Content Drive: Search — #35430

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by measuring current search and pagination on an instance with millions of assets, and review the related Content Drive Search issue #35430. Establish the current response times, investigate whether slowness and incorrect relevance share a cause, and resolve the response-time target and field-precedence decisions before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, search
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.