[VL] Unstable Spark UT
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 85
Description
### Backend
VL (Velox)
### Bug description
https://github.com/apache/gluten/actions/runs/24952706498/job/73065994432?pr=11988
```
2026-04-26T10:23:14.1390962Z - SPARK-16644: Aggregate should not put aggregate expressions to constraints *** FAILED ***
2026-04-26T10:23:14.1392962Z java.lang.NullPointerException: Cannot invoke "org.apache.spark.sql.execution.SparkPlan.subqueries()" because "x$7" is null
2026-04-26T10:23:14.1395298Z at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanHelper.$anonfun$subqueriesAll$1(AdaptiveSparkPlanHelper.scala:121)
2026-04-26T10:23:14.1397684Z at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanHelper.$anonfun$flatMap$1(AdaptiveSparkPlanHelper.scala:75)
2026-04-26T10:23:14.1399606Z at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanHelper.$anonfun$flatMap$1$adapted(AdaptiveSparkPlanHelper.scala:75)
2026-04-26T10:23:14.1401383Z at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanHelper.foreach(AdaptiveSparkPlanHelper.scala:45)
2026-04-26T10:23:14.1402853Z at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanHelper.foreach$(AdaptiveSparkPlanHelper.scala:44)
2026-04-26T10:23:14.1404026Z at org.apache.gluten.test.FallbackUtil$.foreach(FallbackUtil.scala:31)
2026-04-26T10:23:14.1405212Z at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanHelper.flatMap(AdaptiveSparkPlanHelper.scala:75)
2026-04-26T10:23:14.1406736Z at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanHelper.flatMap$(AdaptiveSparkPlanHelper.scala:73)
2026-04-26T10:23:14.1407966Z at org.apache.gluten.test.FallbackUtil$.flatMap(FallbackUtil.scala:31)
2026-04-26T10:23:14.1409251Z at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanHelper.subqueriesAll(AdaptiveSparkPlanHelper.scala:121)
2026-04-26T10:23:14.1410234Z ...
```
### Gluten version
_No response_
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
Contributor guide
Research direction
Start with the linked GitHub Actions job and the SPARK-16644 failure, then inspect FallbackUtil.scala around the call into AdaptiveSparkPlanHelper.subqueriesAll at AdaptiveSparkPlanHelper.scala:121. Reproduce the unstable Spark unit test in the VL backend and trace why a null SparkPlan reaches subqueriesAll. Done means the test no longer fails intermittently and the relevant test run passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, spark
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100