dotCMS / dotCMS/core

Classify OpenSearch startup connection failures (TLS-scheme vs 403 vs connection-refused) for actionable messages

Open
#36,245 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

OpenSearch stale Team : Maintenance Type : Task
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Description

Follow-up to #36244. When the OpenSearch (OS) startup connection gate fails, the failure cause is currently indistinguishable: the connection probe is wrapped in a broad catch (Exception), so a TLS scheme mismatch, an HTTP 403 (auth), and a plain connection-refused all surface as the same opaque error. This produces fatal/fallback messages that don't tell the operator what to actually fix.

This was the diagnostic gap that motivated #36244: the real failure during the 2026-06-12 Phase 3 smoke against the os-migration limited-user stack was a TLS scheme mismatch — dotCMS configured with http://localhost:9201 talking to an OS 3.x port that only accepts https. The server closed the socket → ConnectionClosedException: Connection closed by peer (curl: HTTP 000 / exit 52), which at the catch (Exception) level is indistinguishable from "host unreachable" or a 403.

This issue is intentionally orthogonal to #36244: that issue reworks when/where the gate runs and how it behaves per migration phase (retry → abort vs fallback). This one improves how the failure cause is classified and reported once a connection attempt fails. Keeping them separate keeps each PR focused and reviewable.

Acceptance Criteria

  • On a failed OS connection attempt, the error is classified into at least: TLS/scheme mismatch (e.g. http configured against a TLS-only port → socket closed / ConnectionClosedException), authentication/authorization failure (HTTP 401/403), and unreachable / connection-refused (host down, wrong port, DNS).
  • The fatal (Phase 3 abort) and the WARN/ERROR (Phase 1–2 fallback) messages name the classified cause and the actionable fix (e.g. "OS endpoint scheme mismatch: configured http://… but the server requires https — update OS_ENDPOINTS").
  • Classification is best-effort and degrades gracefully: an unrecognized failure still falls through to a generic "cannot connect to OpenSearch" message — no exception escapes the gate.
  • Classification logic is covered by unit tests for each recognized failure mode.

Priority

Low — diagnostic/observability improvement. It does not change startup gating behavior (that is #36244); it makes failures self-explanatory.

Additional Context

  • Parent: #36244 (unify the OS startup connection gate; phase-aware retry/fallback).
  • The relevant catch (Exception) sites are the OS connection probes exercised by the startup gate: OSIndexAPIImpl.waitUtilIndexReady() / getClusterStats() and IndexStartupValidator.validateOSVersion() (the osClientProvider.getClient().info() probe).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the connection probes in OSIndexAPIImpl.waitUtilIndexReady(), OSIndexAPIImpl.getClusterStats(), and IndexStartupValidator.validateOSVersion(), including the osClientProvider.getClient().info() call and their broad exception handling. Trace how failures reach the startup gate's fatal and WARN/ERROR messages, then add unit coverage for TLS/scheme mismatch, 401/403, connection refusal, and an unknown failure; done means each recognized cause has an actionable message and unknown failures remain generic without escaping.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.