[VL] Strip default comparator from array_sort for Velox offloading
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 85
Description
### Backend
VL (Velox)
### Bug description
When Spark's array_sort is called without an explicit comparator (the default ascending sort), Spark internally
generates a LambdaFunction that wraps the simple ascending comparison in null-handling If/IsNull logic. Velox's
rewriteArraySortCall uses a SimpleComparisonMatcher that cannot parse this null-handling wrapper, causing the
expression to fall back to Vanilla Spark.
### Gluten version
main branch
### Spark version
spark-4.0.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
Contributor guide
Research direction
Start at rewriteArraySortCall and inspect how SimpleComparisonMatcher handles the LambdaFunction generated for array_sort without an explicit comparator. Reproduce the default ascending sort on Spark 4.0.x, then verify that the expression is offloaded to Velox instead of falling back to Vanilla Spark.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100