opensearch-project / opensearch-project/sql
[FEATURE] Improve COUNT function for multivalued fields
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 176
- Forks
- 229
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 43
Description
Is your feature request related to a problem?
When the COUNT function is applied to a multivalued field, it will return the count of all scalar values associated with that field. Consider this table:
x | y
--+----------
1 | {1, 2, 3}
2 | {4, 5}
3 | NULL
COUNT(y) will return 5. Most SQL engines would return 2 in this case, since there are 2 rows containing a value for y.
What solution would you like?
The aggregate requested from the OpenSearch server could be more like:
"aggs": {
"foo": {
"filter": {
"bool": {
"must": [
{
"exists": {
"field": "y"
}
}
]
}
}
}
}
There are a few forms of COUNT that need to be considered:
COUNT(<FIELD>)- can use an aggregate like aboveCOUNT(<LITERAL>)- can use an aggregate like above, but use the field_idto count all rowsCOUNT(*)- can use an aggregate like above, but use the filed_idto count all rowsCOUNT(NULL)- will always return 0
What alternatives have you considered?
The alternative is to keep the existing behaviour. This is likely to produce results that are unexpected for users, especially if they are used to other SQL engines.
Do you have any additional context?
Recently a change was made to support multivalued fields in the SQL plugin. Some of the aggregates may need to be updated to handle the new multivalued fields.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named. Start by locating the SQL plugin's COUNT aggregate handling and the recent multivalued-field support, then trace how field, literal, wildcard, and NULL expressions are represented. Done means COUNT(field) counts rows with a value, COUNT(literal) and COUNT(*) count rows, and COUNT(NULL) returns zero, with coverage for each form.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100