dotCMS / dotCMS/core

Align OpenSearch Site Search index naming with the .os + versioned-store migration pattern

Open
#36,672 1 comment 0 reactions 0 assignees View on GitHub

A pull request for this has already been merged.

  • #36673 by @fabrizzio-dotCMS — merged
OpenSearch Team : Scout Type : Refactoring
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Description

During the Site Search QA of the ES→OpenSearch migration (#36360, fixes in PR #36645) we confirmed a naming inconsistency: the physical OpenSearch Site Search index does not carry the .os suffix, while the rest of the migration does. The .os marker for Site Search is effectively split across two layers:

  • DB / identity (VersionedIndices): tagged. VersionedIndicesAPIImpl.saveIndices force-tags the slot (indices.siteSearch().map(IndexTag.OS::tag)) and the IndicesFactory.requireOSTagged invariant enforces it → the pointer is stored as sitesearch_xxx.os.
  • Physical OpenSearch index: bare. OSSiteSearchAPI.physicalName() applies only the cluster prefix, no .os.
  • Reconciled with patches: OSSiteSearchAPI.defaultSiteSearchIndex() does IndexTag::strip on read, and IndexAPIImpl.providerName carries a carve-out (!IndexType.SITE_SEARCH.is(logical)) so the content lifecycle cascade never tags Site Search names.

The javadoc rationale ("separate clusters") does not hold — content indices are also on separate clusters yet use .os. This is the same bare-vs-.os asymmetry class that previously caused 23+54 CI failures (see the OSIndexAPIImpl asymmetry history). It also has a known failure mode: on a single cluster (the opensearch-upgrade profile, esSameAsOs()), the bare Site Search indices would collide — the code even carries a TODO about this.

Decision (Option A): make Site Search a first-class citizen of the migration naming pattern.

Constraints:

  • (a) The .os tag stays in the DB — the VersionedIndices tagging is not touched.
  • (b) The physical OpenSearch Site Search index must also carry .os.
  • (c) Site Search index rows must use the version field (3.x) to identify them, the same way content indices do.
Implementation notes (touch points)
  • OSSiteSearchAPI.physicalName() + every physical op (createSiteSearchIndex, putMapping, putToIndex, deleteFromIndex, getFromIndex, search, getAggregations, getFacets) must target the .os-tagged physical name.
  • Keep the dual-write list dedup (QA TC-068): OSSiteSearchAPI.listIndices() must still return logical (untagged) names so the ES∪OS merge in SiteSearchAPIImpl.listIndices() keeps deduplicating and never surfaces a .os duplicate. Do the same for anything user-facing (portlet, $sitesearch.listSearchIndicies, alias display).
  • Aliases (createAlias, getIndexAlias, getAliasToIndexMap) and setDefaultToSpecificPosition must stay consistent between the logical (display/merge) and physical (.os) forms.
  • Once the physical index is tagged, remove/adjust the compensations: the IndexTag::strip in defaultSiteSearchIndex() and the Site Search carve-out in IndexAPIImpl.providerName.
  • requireOSTagged already covers site_search — no change needed there.
Risks
  • This is the most fragile area of the migration (the "two-strike lesson" — do not put .os resolution in the wrong layer).
  • Regression risk of double-listing in dual-write phases if listIndices stops deduplicating (QA TC-068).
  • Cannot be fully validated locally — requires the OpenSearch Upgrade Suite in CI.

Acceptance Criteria

  • The physical OpenSearch Site Search index is created and addressed with the .os tag (create/mapping/put/get/delete/search/aggregations all hit the tagged physical name).
  • The .os tag continues to be stored in the DB VersionedIndices store (no change to that tagging), and Site Search rows carry version = 3.x.
  • SiteSearchAPIImpl.listIndices() in a dual-write phase returns each logical index once — no .os duplicate leaks to the portlet or $sitesearch (QA TC-068 still passes).
  • Aliases resolve correctly (valid alias → backing index; unknown alias → clear error) with the tagged physical index.
  • The IndexTag::strip in defaultSiteSearchIndex() and the Site Search carve-out in IndexAPIImpl.providerName are removed or reduced to the minimum still required, with no behavior change for search/aggregations/facets.
  • Single-cluster (esSameAsOs()) dual-write no longer collides on Site Search index names.
  • OSSiteSearchAPIIntegrationTest updated + new coverage for dual-write listIndices dedup; the OpenSearch Upgrade Suite is green in CI.

Priority

Medium

Additional Context

  • Follow-up from #36360 (Site Search QA) and PR #36645 (the QA fixes I-3/I-5/I-6/I-7 already merged/queued there — this issue intentionally does not bundle the naming refactor into that PR).
  • Related: #35786 (vendor-neutral Site Search + OS impl), #35640 (index delete guard + .os cascade), and the OSIndexAPIImpl bare-vs-.os asymmetry history.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with OSSiteSearchAPI.physicalName() and its physical operations, then read SiteSearchAPIImpl.listIndices(), defaultSiteSearchIndex(), and IndexAPIImpl.providerName. Run OSSiteSearchAPIIntegrationTest and inspect QA TC-068 coverage before changing behavior. Done means tagged physical indices, version 3.x rows, deduplicated logical listings, consistent aliases, and passing OpenSearch Upgrade Suite coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, databases, search
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.