dotCMS / dotCMS/core

Integration harness never starts a ReindexThread, so full-reindex switchover has no automated coverage

Open
#37,302 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Team : Maintenance Type : Test Automation
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Summary

The integration-test harness never starts a ReindexThread, so nothing drains the reindex queue. Any integration test that depends on the queue draining on its own — most importantly a full reindex reaching switchover — cannot pass, and there is currently no automated coverage of that path.

Evidence

Observed while adding integration coverage for #36922 (PR #37295). Full-log run of a test that calls fullReindexStart() and waits for the switchover:

  • The reindex starts correctly — reindex_working is populated:
    16:44:08.954  IndiciesAPIImpl - Indices configuration updated ... reindex_working: cluster_d0d0578ec2.working_20260831164408
    
  • The log then contains zero ReindexThread lifecycle lines — no ReindexThread is starting, no Recreating ReindexThread, no Unpausing reindex thread, no ReindexThread Paused — and zero Running Reindex Switchover lines.
  • At 16:48:09, exactly 240s later, reindex_working: null appears. That is the test's own fullReindexAbort() in finally, not a switchover.

So the rebuild queue is never drained, finalizeReIndex() is never reached, and switchOverIfNeeded() is never called.

Cause

ReindexThread.startThread() delegates to unpause(), which — unless ALLOW_MANUAL_REINDEX_UNPAUSE is set — only registers a Hibernate commit listener:

HibernateUtil.addCommitListener("unpauseIndex", ReindexThread::unpauseImpl);

With no committing transaction in the harness bootstrap, the listener never fires and the worker never starts. DotCMSInitDb does call startThread(), but that call has the same commit-listener indirection.

This also explains why sibling tests in the same class do pass: saving a contentlet commits a transaction, which fires the listener as a side effect and incidentally starts the worker. Coverage of the reindex queue is therefore accidental rather than arranged.

Not a duplicate of #37281 / #37282

Both of those concern the switchover being deferred or blocked by the minimum-runtime guard in fullReindexSwitchover(), whose signature is Running Reindex Switchover / Reindex Time Elapsed not set. logged every 3 seconds. This run logs none of those — the switchover is never attempted, because the worker never runs. If #37282 is fixed, this gap remains.

Not a regression either: the same test fails identically (244.0s, same assertion) against unmodified main. The same operation succeeds in ~30s on a real server, so the product path is healthy.

Suggested fix

Have the integration harness start the reindex worker deterministically rather than as a side effect of an unrelated commit — for example by setting ALLOW_MANUAL_REINDEX_UNPAUSE=true in the IT configuration so unpause() calls unpauseImpl() directly, or by having IntegrationTestInitService start it explicitly.

Ready-made regression test

PR #37295 adds ReindexThreadTest#test_full_reindex_completes_and_switches_over, currently @Ignore-d with this diagnosis in its Javadoc. It asserts that a full reindex completes and the switchover happens (polling, bounded timeout). Re-enable it to verify this issue is fixed — no new test needs to be written.

Acceptance Criteria

  • The integration harness starts a ReindexThread deterministically, not as a side effect of an unrelated transaction commit
  • ReindexThreadTest#test_full_reindex_completes_and_switches_over passes with its @Ignore removed
  • The queue-draining path has automated coverage that does not depend on incidental commits

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ReindexThread.startThread() and its unpause() behavior, then inspect IntegrationTestInitService and the integration-test configuration for how the worker is started. Re-enable ReindexThreadTest#test_full_reindex_completes_and_switches_over from PR #37295 and run it. Done means the harness starts the worker deterministically and the full reindex reaches switchover without relying on an unrelated commit.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.