ES→OS: endpoint to audit and rewrite VTL that calls the deprecated $estool.esSearch / $estool.esRaw
@fabrizzio-dotCMS is already working on this.
Since Sep 11, 2026.
- Dominant language
- Java
- Stars
- 970
- Forks
- 486
- Avg merge
- 3d 33m
- Merged PRs (30d)
- 170
Description
Description
Customers migrating from Elasticsearch to OpenSearch 3 have to find and rewrite every Velocity
template that calls the two deprecated search methods, $estool.esSearch(...) and
$estool.esRaw(...). Today this is entirely manual, and it is the single hardest part of the
migration for a customer to do alone:
- The code is spread across four places — templates and containers stored in the database, file
templates/containers in the assets tree, widget code and custom-field Velocity inside content type
definitions, and Velocity script steps inside workflow actions. Nothing but the files can be
grepped. - The failure is silent. These templates keep working through Phase 1 (reads still come from
Elasticsearch) and break in Phase 2, and they break by rendering less, not by erroring.
This task adds one role-gated endpoint that both reports what needs changing and, when explicitly
instructed, applies the mechanical part of the change with a manifest that can be reverted.
Endpoint modes
| Mode | Behaviour |
|---|---|
GET /api/v1/index/migration/vtl-audit |
Read-only report: object type, id, name, line, rule, confidence. Touches nothing. |
POST /api/v1/index/migration/vtl-audit/_apply with dryRun: true |
Returns the exact diff it would write. Writes nothing. |
POST /api/v1/index/migration/vtl-audit/_apply |
Writes the working version only of each object — never publishes — and returns a runId plus the manifest (object, rule applied, previous version). |
POST /api/v1/index/migration/vtl-audit/_revert with a runId |
Restores the previous version of every object in that manifest. |
Rules
Auto-rewritable (high confidence):
| From | To |
|---|---|
$estool.esRaw( |
$estool.raw( |
$r.took (on a variable bound to a raw response) |
$r.tookInMillis |
$r.hits.getAt(n) |
$r.hits.hits.get(n) |
$estool.esSearch( and the $item.map.field uses of its loop variable |
$estool.search( and $item.field |
Reported, never rewritten:
| Found | Why it needs a human |
|---|---|
$r.toString() used as JSON |
The neutral response does not serialize to engine JSON; the JSON has to be rebuilt field by field. Highest-risk finding: it fails silently. |
$results.hits.maxScore |
No equivalent on the neutral response; the score has to be read per hit. |
esSearch whose result variable escapes the file (#parse, #dotParse, a macro) |
The rewrite of .map. uses cannot be attributed safely. |
| Velocity steps in workflow actions | Reported with the action id; rewriting is allowed only with an explicit target list. |
Design constraints
- Only rewrite what is provably local. If the result variable of an
esSearchcall crosses a
#parse/#dotParse/ macro boundary, the finding is reported, not rewritten. A wrong rewrite
here fails the same silent way the migration itself does. - Never publish. Writing the working version keeps the live site untouched and uses the
versioning that templates, containers, content types and file assets already have — restoring the
previous version is the rollback. Publishing stays a human decision after reading the diff. - The manifest is the only backup we own. Workflow actionlets do not version, so for those the
previous code is stored whole in the manifest. - Idempotent. A second run finds nothing to do;
_revertworks as long as therunIdmanifest
exists. - Gated and off by default. Same gate as the readiness endpoint — CMS Administrator plus the
migration role — since this reads and rewrites customer code.
Acceptance Criteria
-
GET .../vtl-auditreturns findings from all four sources: database templates and containers,
file templates/containers, content type field values (widget code and custom fields), and
Velocity workflow actionlets. - Every finding carries object type, id, human-readable name, line, rule, and whether it is
auto-rewritable or manual. -
_applywithdryRun: truereturns a diff and provably writes nothing. -
_applywrites only working versions; nothing is published, and the live site renders
identically immediately after a run. -
_applyreturns arunIdwhose manifest names every object touched and its previous version. -
_revertwith thatrunIdrestores every object to its previous state, including workflow
actionlets, which have no versions of their own. - Re-running
_applyon an already-converted installation reports and changes nothing. - A template whose
esSearchresult escapes through#parseor a macro is reported and left
untouched. -
toString()-as-JSON andmaxScorefindings are never rewritten. - Both endpoints answer 403 without CMS Administrator plus the migration role.
- Integration tests cover a fixture with one case per rule, plus an already-migrated fixture that
must produce zero findings.
Additional Context
The rules come from the customer-facing migration guide's template section, which was verified
against the code:
ESContentTool.esSearchreturnsESSearchResultsholding rawContentlets;ESContentTool.search
returnsContentSearchResults<ContentMap>.ContentMaphas nogetMap(), which is why
$item.map.fieldstops resolving after the rename — silently.ContentSearchResponseexposesgetTookInMillis(), andSearchHitsexposesgetHits()/
getTotalHits(); the oldgetAt(n)andgetMaxScore()have no counterpart.- On Elasticsearch the neutral
search()delegates to the sameesSearch()underneath
(ESSearchAPIImpl.search), so query handling is unchanged by the rewrite — only the result type is.
Both deprecated methods are marked @Deprecated(forRemoval = true) for removal in v26.08.04.
Related: the deprecation inventory in docs/backend/ES_OS_DEPRECATIONS.md and the plugin audit skill
requested in #37466 — this issue is the Velocity half, and the two should share one rule source
rather than each keeping its own list.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.