[Python][Substrait] Acero consumer is unable to consume count function from substrait query plan
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
SQL
```java
SELECT
o_orderpriority,
count(*) AS order_count
FROM
orders
GROUP BY
o_orderpriority
```
The substrait plan generated from SQL, using Isthmus.
substrait count:
Running the substrait plan with Acero returns this error:
```java
E pyarrow.lib.ArrowInvalid: JsonToBinaryStream returned INVALID_ARGUMENT:(relations[0].root.input.aggregate.measures[0].measure) arguments: Cannot find field.
```
From substrait query plan:
relations[0].root.input.aggregate.measures[0].measure
```java
"measure": {
"functionReference": 0,
"args": [],
"sorts": [],
"phase": "AGGREGATION_PHASE_INITIAL_TO_RESULT",
"outputType": {
"i64": {
"typeVariationReference": 0,
"nullability": "NULLABILITY_REQUIRED"
}
},
"invocation": "AGGREGATION_INVOCATION_ALL",
"arguments": []
}
```
```java
"extensions": [{
"extensionFunction": {
"extensionUriReference": 1,
"functionAnchor": 0,
"name": "count:opt"
}
}],
```
Count is a unary function and should be consumable, but isn't in this case.
**Reporter**: [Richard Tia](https://issues.apache.org/jira/browse/ARROW-17061) / @richtia
#### PRs and other links:
- [GitHub Pull Request #14123](https://github.com/apache/arrow/pull/14123)
**Note**: *This issue was originally created as [ARROW-17061](https://issues.apache.org/jira/browse/ARROW-17061). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Reproduce the SQL aggregation with the shown Substrait plan and inspect Acero's Substrait plan conversion around relations[0].root.input.aggregate.measures[0].measure. Compare the count:opt extension and its zero-argument measure with the consumer's expected representation. Done means the plan is accepted and the grouped count query executes without the Cannot find field error; PR #14123 is also relevant.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100