opensearch-project / opensearch-project/sql
[BUG][Umbrella] Array values are not treated as atomic values in SQL/PPL operations
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 176
- Forks
- 229
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 43
Description
What is the bug?
After #1300 enabled array values in query evaluation and result sets, the engine decomposes arrays into individual scalar elements for operations that should treat the array as a single unit. This causes incorrect results or exceptions across multiple SQL/PPL operations.
Affected operations
| Operation | Symptom | Issue |
|---|---|---|
WHERE comparisons (=, !=, >, >=, <, <=) |
If any element passes, the whole array passes the filter | #3136 |
| ORDER BY | Only the first element is used for comparison | #3133 |
| GROUP BY | Each array element is treated as a separate group key | #3132 |
| MAX/MIN aggregation | Operates on individual elements across all arrays instead of comparing whole arrays | #3138 |
regex command |
Only matches patterns against the first element of an array field | #4461 |
| Calcite eval/arithmetic expressions | ClassCastException: ArrayList cannot be cast to Long — no array handling at all |
#5232 |
How to reproduce
Create an index with array-valued fields:
{"x": 1, "y": [1, 2]}
{"x": 2, "y": [3, 4]}
{"x": 3, "y": [1, 5]}
{"x": 4, "y": [1, 2]}
{"x": 5, "y": [2, 3]}
Then run any of the following:
-- WHERE: returns rows where ANY element > 3, not treating array as atomic
SELECT x, y FROM test WHERE y > 3;
-- ORDER BY: sorts by first element only
SELECT x, y FROM test ORDER BY y;
-- GROUP BY: explodes array into individual group keys
SELECT COUNT(x), y FROM test GROUP BY y;
-- MAX/MIN: returns max/min of all individual elements, not max/min array
SELECT MAX(y) FROM test;
Expected behavior
Arrays should be treated as indivisible values. Operations on array vs scalar should either:
- Throw an exception (type mismatch), or
- Have well-defined semantics (e.g., element-wise comparison for array vs array)
Root cause
The engine does not distinguish between array values and scalar values at the operator level. When an array is encountered, it is implicitly flattened or only the first element is used, depending on the code path.
Duplicates
This issue consolidates the following duplicates:
- #3132 — GROUP BY with array values
- #3133 — ORDER BY with array values
- #3136 — Comparison operators with array values
- #3138 — MAX/MIN with array values
- #4461 — regex matching on array fields
- #5232 — ClassCastException on multi-valued long fields
- #3206 — Parse on multi-valued fields
- #4365 — Pattern on multi-valued fields
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the affected-operation issues listed in the table, especially #3132, #3133, #3136, #3138, #4461, and #5232, then run the SQL examples against array-valued fields. Done means the project defines and consistently applies atomic-array behavior for the listed WHERE, ORDER BY, GROUP BY, aggregation, regex, and expression paths without flattening or array-related exceptions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100