gluten-ut: checkConsistencyBetweenInterpretedAndCodegen never reaches the backend
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### What happens
`checkConsistencyBetweenInterpretedAndCodegen` looks like the strongest assertion in a Gluten expression test, and it never reaches the backend. Example at `gluten-ut/spark35/src/test/scala/org/apache/spark/sql/catalyst/expressions/GlutenMathExpressionsSuite.scala:298`:
```scala
checkConsistencyBetweenInterpretedAndCodegen((e: Expression) => Unhex(e), StringType)
```
`GlutenTestsTrait` overrides `checkEvaluation`, `checkExceptionInExpression` and `checkResult`. It does not override this one. The helper comes from Spark's `ExpressionEvalHelper` and goes through `evaluateWithoutCodegen` plus `GenerateMutableProjection`, never building a DataFrame, so no Gluten plan is produced and Velox is not involved.
### Why it matters
What the line actually compares is vanilla Spark interpreted evaluation against vanilla Spark codegen. If Velox's `unhex` were completely broken, the line would still pass. Because it generates random literals it reads like the broadest check in the case, so it is easy to mistake for a Velox consistency guarantee, and easy to conclude a Gluten expression is well covered when it is not.
This is inherited from Spark's own suites rather than written for Gluten, so it is likely to appear in other `Gluten*ExpressionsSuite` files. Worth a sweep rather than a one-line fix.
### Suggested fix
Either replace it with generated literals driven through `checkEvaluation`, which does go through Gluten, or keep it and add a comment saying it only covers vanilla consistency. The first option is the useful one, provided somebody checks the generated inputs are ones the backend supports.
Found during a review pass on #12840. That PR touched this line only to adapt it to Spark 3.4's two-argument `Unhex`, and left the semantics alone on purpose.
Contributor guide
Research direction
Start with gluten-ut/spark35/src/test/scala/org/apache/spark/sql/catalyst/expressions/GlutenMathExpressionsSuite.scala:298 and inspect GlutenTestsTrait's checkEvaluation, checkExceptionInExpression, and checkResult overrides. Trace ExpressionEvalHelper's checkConsistencyBetweenInterpretedAndCodegen through evaluateWithoutCodegen and GenerateMutableProjection, then sweep other Gluten*ExpressionsSuite files. Done means affected checks either exercise the backend through generated inputs or are explicitly documented as vanilla-only.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- backend, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100