Readiness report prose contradicts its own data: healthy indexes called leftovers at phase 0, wrong drift direction at phase 3
@fabrizzio-dotCMS is already working on this.
Since Sep 18, 2026.
- Dominant language
- Java
- Stars
- 970
- Forks
- 486
- Avg merge
- 3d 33m
- Merged PRs (30d)
- 170
Description
Found during a lab run of the ES→OpenSearch 3 migration, by Jamie Mauro.
Description
The readiness report's numbers are right; the sentences it wraps around them are not. Two instances,
both in MigrationReadinessService.evaluate(), both reassuring the reader about something other than
what is actually true.
1. Phase 0 calls healthy indexes "left over from an earlier migration attempt"
Rolled back to phase 0, the summary read:
Note: 2 indices from an earlier migration attempt are left over on OpenSearch; dual-write will
overwrite them on the next crawl/reindex.
Those were the current, in-sync OpenSearch indexes, built over the preceding 22 hours at phase 1 and
verified at 3 and 5 documents — matching Elasticsearch exactly. The report sees OpenSearch indexes
while at phase 0, concludes they must be debris, and says so.
An operator who drops to phase 0 to troubleshoot — which the guide presents as a safe move — is told
their good indexes are junk.
The second clause is wrong on its own terms too: dual-write does not backfill. Only a reindex
would repair those indexes if they genuinely were stale. Promising that dual-write will overwrite them
contradicts the rule the rest of the migration is built on.
The code is at MigrationReadinessService.java:~102. The comment above it shows the intent — "anything
still counted here is NOT a yet-to-be-built counterpart" — but a temporary rollback from a dual-write
phase produces exactly that state and is not distinguished from real debris.
2. Phase 3 volunteers the wrong drift direction
With OpenSearch missing two documents that Elasticsearch had:
"outOfSyncCount": 2,
"summary": "Phase 3 (OpenSearch only) — the final phase, nothing to advance to.
No index shows Elasticsearch behind OpenSearch; still verify before any downgrade."
The actual drift was OpenSearch behind Elasticsearch. The summary is technically true — it reports
on the other direction — but it volunteers a reassurance about the direction that does not matter
here and says nothing about the gap that does.
At the terminal phase, where OpenSearch is the only engine serving traffic, a bare outOfSyncCount: 2
is the sole hint that live search is missing content. The code branches only on esBehindAnywhere
(MigrationReadinessService.java:~82-87), so the opposite condition has no sentence of its own.
Why this matters
The operator documentation tells the reader to trust this report over document counts and the admin
UI. Both of these send a reader in the wrong direction while the underlying data was correct all
along — which is the worst shape for a diagnostic to fail in, because the numbers next to the prose
look authoritative.
Acceptance Criteria
- At phase 0, OpenSearch indexes that are in sync with their Elasticsearch counterparts are not
described as leftovers from an earlier attempt. - No summary claims dual-write will repair a stale index; the repair is a reindex.
- At phase 3, a summary that mentions drift direction addresses the direction actually present —
and OpenSearch being behind Elasticsearch gets its own sentence, since that is the one affecting
live traffic. - Tests cover the phase-0-after-rollback state and the phase-3 OS-behind state.
Additional Context
Same lab run as #37635, #37636 and #37637. Related in kind to #37635, where the report's prose reports
health while its content map is empty — this issue is the same failure mode with the data present.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.