dotCMS / dotCMS/core

Content Search Pagination displays incorrect counts due to ES index vs database mismatch

Open
#33,060 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

OKR : Customer Support Team : Maintenance Type : Defect
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Problem Statement

Content list pagination displays incorrect counts (e.g., "Showing 1 - 40 of 43" when only 40 items exist, or showing pagination for page 2 when no results exist on page 2). The total count comes from Elasticsearch, but actual results come from database lookups, creating a mismatch.

Steps to Reproduce

Navigate to the content list view in the admin interface
Observe pagination showing "Showing 1 - X of Y"
Note that Y (total count) is higher than the actual visible content
Navigate to page 2 - shows "Showing X-Y of Z," but the page may be empty
Reindexing and ES optimization do not resolve the issue permanently

Observed only on Client env's

We could re-produce this in demo by creating some orphanage data in the contentlent_version_info table

Acceptance Criteria

Pagination should show accurate counts: "Showing 1 - 4 of 4" when four items exist
Page 2 should only be available when more than 1 page of results exists
Counts should be consistent after reindexing

dotCMS Version

Evergreeen

Proposed Objective

Core Features

Proposed Priority

Priority 3 - Average

External Links... Slack Conversations, Support Tickets, Figma Designs, etc.

https://dotcms.freshdesk.com/a/tickets/32700

Assumptions & Initiation Needs

Two-Stage Search Process Creates Mismatch:

Stage 1 - Elasticsearch Count
Stage 2 - Database Lookup

Why Counts Differ:
ES Permission Query: Uses broad permission patterns that may match non-existent content
Database Filtering: findContentlets() filters out missing/invalid contentlets:

  return inodes.stream().map(inode -> conMap.get(inode))
      .filter(Objects::nonNull)  // ← Removes null contentlets
      .collect(Collectors.toList());

Orphaned entries in contentlet_version_info table without corresponding contentlet records
Permission index entries for deleted content
ES index contains references to contentlets that no longer exist in database

Quality Assurance Notes & Workarounds

No response

Sub-Tasks & Estimates

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the admin content list view and trace the two-stage search flow described in the issue: the Elasticsearch count and the database lookup through findContentlets(). Inspect how orphaned contentlet_version_info rows and missing contentlets affect the returned count. Done means pagination counts match visible results, page 2 is unavailable when empty, and counts remain correct after reindexing.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, java
Domain
backend, databases, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.