[OpenSearch Migration] Phase 3: inferIndexToHit throws when VersionedIndices is unpopulated; write vs read/count paths resolve the index from divergent sources

Summary

Surfaced while working on PR #36027 (#36026 VTL aggregation regression). In Phase 3 (OS-only, FEATURE_FLAG_OPEN_SEARCH_PHASE=3) the OpenSearch read/count path throws when the new VersionedIndices (OPENSEARCH_3X) rows are not populated, even though content was written successfully — because the write path and the read/count path resolve the target index from different sources.

Evidence

ContentFactoryIndexOperationsOS.inferIndexToHit (dotCMS/src/main/java/com/dotcms/content/index/opensearch/ContentFactoryIndexOperationsOS.java:360-385):

final Optional<VersionedIndices> optional = Try.of(() ->
    APILocator.getVersionedIndicesAPI().loadDefaultVersionedIndices()).getOrElse(Optional.empty());
if (optional.isEmpty()){
    throw new DotRuntimeException("Unable to load default versioned indices, falling back to default index");
}

Two problems:

1. Misleading message / no fallback. The message says "falling back to default index" but the code throws — it never falls back.

2. Divergent index resolution in Phase 3. Observed in an integration run (Phase 3):

   • Write path (addContentToIndex → publish) resolved and wrote to the bootstrap content index live_<timestamp> (via the legacy IndiciesInfo/IndiciesAPI) — succeeded.
   • Read/count path (isInodeIndexed → ESContentletAPIImpl.indexCount → ContentFactoryIndexOperationsOS.inferIndexToHit) used VersionedIndicesAPI.loadDefaultVersionedIndices() (OPENSEARCH_3X), which was empty → threw DotRuntimeException: Unable to load default versioned indices, falling back to default index.

   Stack:

   ContentFactoryIndexOperationsOS.inferIndexToHit(ContentFactoryIndexOperationsOS.java:365)
   ContentFactoryIndexOperationsOS.getCountRequest(...:400)
   ContentFactoryIndexOperationsOS.indexCount(...:392)
   ESContentFactoryImpl.indexCount(ESContentFactoryImpl.java:1580)
   ESContentletAPIImpl.isInodeIndexed(ESContentletAPIImpl.java:9829)
   

Nothing in the opensearch-upgrade test environment's bootstrap populates the OS VersionedIndices for the default content indices, so any Phase-3 read/count that flows through inferIndexToHit fails.

Impact

• Any Phase-3 code path that goes through inferIndexToHit (e.g. isInodeIndexed, index counts) throws if OS VersionedIndices haven't been registered, while writes silently use a different (legacy) resolution source. This write/read divergence is a latent correctness/robustness gap for Phase 3.

Suggested direction

• Make inferIndexToHit actually fall back (and/or unify index resolution so write and read/count paths use the same source), or ensure the bootstrap/activation flow populates OS VersionedIndices whenever content indices are created in Phase 3. Fix the misleading exception message regardless.
• Likely related to the OS index-naming work on fix/issue-35820-os-index-naming-suffix.

How it was found

Adding ContentSearchToolTest to OpenSearchUpgradeSuite (PR #36027) drove the full Phase-3 content lifecycle end-to-end and exposed this. That test has since been moved to the standard ES suite (MainSuite1b); OS-3.x aggregation coverage remains via OSSearchAPIImplIntegrationTest.

Start in dotCMS/src/main/java/com/dotcms/content/index/opensearch/ContentFactoryIndexOperationsOS.java, especially inferIndexToHit, and trace addContentToIndex alongside getCountRequest and indexCount. Run the OpenSearchUpgradeSuite or the cited ContentSearchToolTest flow to reproduce Phase 3 with unpopulated VersionedIndices. Done means read/count resolution no longer throws or diverges from writes, and the misleading fallback message is corrected.