apache / apache/datafusion-comet
feat: Support `array<binary>` for `array_reverse`
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
### What is the problem the feature request solves?
Followup on #2612
We observe incorrect behavior for `array_reverse` if datatype is `array`. Currently we fallback to Spark and we need to support `array` for native execution
Repro issue
```
test("native reader - array_reverse") {
withSQLConf(
CometConf.COMET_EXEC_ENABLED.key -> "true",
SQLConf.USE_V1_SOURCE_LIST.key -> "parquet",
CometConf.COMET_ENABLED.key -> "true",
CometConf.COMET_EXPLAIN_FALLBACK_ENABLED.key -> "true",
CometConf.COMET_NATIVE_SCAN_IMPL.key -> "native_iceberg_compat") {
testSingleLineQuery(
"""
|SELECT array(
| cast(null as array),
| cast(array() as array)
|) AS a
|""".stripMargin,
"select reverse(a) from tbl", debugCometDF = df => df.printSchema())
}
}
!== Correct Answer - 1 == == Spark Answer - 1 ==
struct>> struct>>
![ArrayBuffer(WrappedArray(), null)] [ArrayBuffer(WrappedArray(), WrappedArray())]
```
https://github.com/apache/datafusion-comet/issues/2612#issuecomment-3508492517
### Describe the potential solution
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.