PhaseRouter reports a malformed query as an OpenSearch availability failure and falls back to Elasticsearch

Found during a lab run of the ES→OpenSearch 3 migration, by Jamie Mauro.

Description

A template sent a query that is not valid JSON. PhaseRouter reported it as an OpenSearch
availability problem and fell back to Elasticsearch:

ERROR index.PhaseRouter - OS read failed in Phase 2 — falling back to ES.
      OS index may be stale or unavailable. Cause: Unable to parse the given query.

The actual cause is client-side and visible in the message itself —
OSSearchAPIImpl.searchRaw does new JSONObject(query) and throws DotStateException("Unable to parse the given query.") when the input is not JSON. Nothing was wrong with the cluster.

Two problems follow from that.

Wrong diagnosis in the log. An operator reading "OS index may be stale or unavailable" goes
looking for a cluster fault. There isn't one; a template sent a bad query. During a migration, when
people are already primed to suspect the new engine, this is an expensive misdirection.

The fallback fires for the wrong class of error. Phase-2 fallback exists so that a degraded
OpenSearch does not take the site down. A 4xx-class client error is not that: the same malformed query
fails identically against the other engine, as it did here — the request still aborted, after doing
the work twice.

Suggested direction

PhaseRouter should separate transport and availability failures from request-validity failures.
Only the former should trigger the fallback and the "stale or unavailable" wording; the latter should
propagate with the underlying message intact.

A reasonable dividing line: DotStateException raised by query parsing (and anything else that means
"the caller sent something invalid") is not a fallback condition.

Acceptance Criteria

• A malformed query at phase 2 does not trigger the Elasticsearch fallback.
• Its log line names the real cause and does not suggest the index is stale or unavailable.
• A genuine connectivity or availability failure still falls back, and still logs as it does today.
• Tests cover both branches, so the classification cannot silently collapse back into one.

Additional Context

Surfaced alongside #37635 and #37636 in the same lab run. Lower severity than those two — it wastes an
operator's time rather than hiding a problem — but it is in the code path the migration leans on most.