PPL (Calcite/v3): valid Unicode string literals and invalid function arguments return HTTP 500 instead of 4xx

Summary

Two categories of PPL query currently fail with HTTP 500 on the Calcite (v3) engine, when the correct behavior is either success (valid input) or a client error (invalid input):

1. Valid Unicode string literals are rejected. A PPL query containing a non-Latin-1 string literal (CJK, emoji, most non-Latin scripts) fails during query validation with a Calcite charset-encoding error and surfaces as HTTP 500. These are legitimate string values and the query should succeed.
2. Invalid function arguments are miscategorized as server errors. A PPL query with an out-of-range function argument (e.g. a negative substring length) fails at execution with a SQLException and surfaces as HTTP 500. This is invalid user input and should be an HTTP 400.

Both stem from exception types escaping to the REST layer without being classified as client errors.

Environment

• Engine: Calcite / v3 (plugins.calcite.enabled=true), default settings (plugins.calcite.fallback.allowed=false).
• Reproduced against current main.

Reproduction

Case A — Unicode string literal (expected: success; actual: 500)

POST /_plugins/_ppl
{ "query": "source=my_index | eval label = '⚠️ 未登録' | fields label" }

Any CJK/emoji/non-Latin-1 literal triggers it, e.g. eval x = '未登録' or where name = '💡'.

• Expected: query succeeds and returns the literal value (HTTP 200).
• Actual: HTTP 500. Underlying error: CalciteException/CalciteContextException — "Failed to encode '…' in character set 'ISO-8859-1'", thrown during validation.

Case B — negative substring length (expected: 400; actual: 500)

POST /_plugins/_ppl
{ "query": "source=my_index | eval s = substring(name, 5, -1) | fields s" }

• Expected: HTTP 400 (invalid argument).
• Actual: HTTP 500. Underlying error: java.sql.SQLException — "Substring error: negative substring length not allowed", thrown at execution.

Root cause

Status classification (both cases)

In plugin/src/main/java/org/opensearch/sql/plugin/rest/RestPPLQueryAction.java the failure listener maps exceptions to HTTP status: getRawErrorCode unwraps the error cause, honors OpenSearchException.status(), and otherwise returns 400 only when isClientError matches — else 500. isClientError recognizes only: IllegalArgumentException, IndexNotFoundException, QueryEngineException, SyntaxCheckException, DataSourceClientException, IllegalAccessException.

The exceptions that actually escape for these two queries — a raw CalciteException/CalciteContextException (case A) and a RuntimeException wrapping SQLException (case B) — are not in that set, so both fall through to 500. On default settings there is no v2 fallback for these errors, so the 500 is deterministic.

Case A specifically — charset

The Calcite engine is initialized without a string charset, so it uses Calcite's global default ISO-8859-1 for both string literals and string columns (framework config uses SqlParser.Config.DEFAULT with no charset override; the Calcite JDBC connection sets no charset property; no calcite.default.charset is configured). Any literal with a code point outside Latin-1 cannot be encoded and validation throws before the query can run.

Proposed fix

• Case A (lead fix): make Unicode literals work. Configure the Calcite engine to use a Unicode charset (UTF-16LE, Calcite's canonical Unicode charset) for string literals and string columns, applied uniformly so mixed-charset comparisons are not introduced. This removes the error entirely so the query succeeds, rather than merely returning a nicer error code.
  • Caveat to verify during implementation: apply the charset consistently to avoid Calcite "Cannot apply '=' to the two different charsets …" comparison errors, and check ORDER BY/comparison collation behavior on strings.
• Case B: classify invalid arguments as 400. Treat out-of-range function-argument failures (e.g. negative substring length) as client errors — preferably by raising/propagating a client-error exception type at the point of failure (the existing QueryEngineException is already mapped to 400 by isClientError), rather than widening the REST-layer catch-all.
• Scope carefully: do not blanket-map all CalciteException/SQLException/RuntimeException to 400 — genuine planner, internal, or I/O failures must remain 500. Classification should target user-input-caused failures specifically.

References (current main)

• plugin/src/main/java/org/opensearch/sql/plugin/rest/RestPPLQueryAction.java:50-59 (isClientError), :62-75 (getRawErrorCode), :133-134 (failure listener).

Start in plugin/src/main/java/org/opensearch/sql/plugin/rest/RestPPLQueryAction.java, especially isClientError, getRawErrorCode, and the failure listener. Reproduce both POST /_plugins/_ppl cases, then trace the Calcite charset configuration and the negative substring failure to their escaping exception types. Done means Unicode literals return HTTP 200 and invalid function arguments return HTTP 400 without reclassifying internal failures.