Reindex index-name timestamp and switchover elapsed-time parse disagree, deferring or blocking the switchover

Summary

The reindex switchover decides whether enough time has elapsed by re-parsing a timestamp out of the reindex index's name on every attempt. On a single node with a correct clock, the name can be stamped ahead of the index's real creation time, making the computed elapsed time negative and blocking the switchover until the wall clock catches up.

Affected version

26.08.03-01 (code path unchanged on main at time of writing)

Observed

Single-node instance, container TZ=UTC, clock correct and agreeing with the host:

container UTC : Fri Aug 28 17:19:48 UTC 2026
index created : cluster_<clusterId>.working_20260828175634

The index name encodes 17:56:34 — roughly an hour ahead of when it was actually created. The switchover panel showed Time : n/a, the indices sat in Building, and the log emitted Running Reindex Switchover / Reindex Time Elapsed not set. every 3 seconds until real time passed the stamped value.

The same artifact was observed on a separate two-node instance: index created at 12:20:11 UTC, named ...20260820132011, switchover deferred ~52 minutes and then logged Reindex took : 2m 2.167s — a duration derived from the misstamped name rather than the real elapsed time.

Code

Elapsed time is derived from the index name, not tracked:

// ContentletIndexAPIImpl.reindexTimeElapsedInLong
final IndiciesInfo oldInfo = legacyIndiciesAPI.loadIndicies();
if (oldInfo.getReindexWorking() != null) {
    return oldInfo.getIndexTimeStamp(IndexType.REINDEX_WORKING);
}

// IndiciesInfo
public static final SimpleDateFormat timestampFormatter = new SimpleDateFormat("yyyyMMddHHmmss");

public long getIndexTimeStamp(final IndexType indexType) {
    final String indexName = indiciesNames.get(indexType);
    final String indexTimestamp = indexName.substring(indexName.lastIndexOf("_") + 1);
    startTime = timestampFormatter.parse(indexTimestamp);
    return System.currentTimeMillis() - startTime.getTime();
}

public String createNewIndiciesName(final IndexType... indiciesType) {
    final String timeStamp = timestampFormatter.format(new Date());
    ...
}

Two problems with that shared static formatter:

1. SimpleDateFormat captures TimeZone.getDefault() at construction. dotCMS mutates the JVM default timezone at runtime — StartupLogger:23 (TimeZone.setDefault(companyTimeZone)), CompanyManagerUtil:207, and DBTimeZoneCheck:57/:77, which sets a candidate zone and restores it in a finally. A static final formatter initialised inside any of those windows keeps that zone for the life of the JVM, so format() and parse() can end up on different offsets.

2. SimpleDateFormat is not thread-safe, and this one is shared between format() (naming the index) and parse() (the switchover gate) — the latter called on every bulk flush via BulkProcessorListener.beforeBulk → reindexTimeElapsed(), from a different thread than the reindex start. Concurrent format/parse corrupts the shared internal Calendar.

A second, separate static SimpleDateFormat with the same pattern exists on the ContentletIndexAPI interface, compounding the divergence risk.

Failures are also invisible: reindexTimeElapsedInLong() swallows every exception and returns 0, logged only at Logger.debug, and reindexTimeElapsed() returns empty for any value <= 0 — surfacing as Time : n/a with no explanation.

Impact

The gate cannot open while elapsed is negative, so the switchover is deferred by however far the name is skewed. Combined with #37281 (a refused forced switchover stays armed), this converts an operator action into an index promotion that fires at an arbitrary later time.

Suggested fix

• Persist the reindex start time rather than re-deriving it from the index name.
• If the name must remain the source of truth, use java.time / DateTimeFormatter with an explicit fixed zone (UTC), which is immutable and thread-safe.
• Do not swallow the parse failure — a gate that can never open should say why above DEBUG.

Related

• #37281 — refused forced switchover stays armed and fires later
• #37279 — the control that requests the switchover

Related Freshdesk ticket: https://helpdesk.dotcms.com/a/tickets/38957