apache / apache/gluten

gluten-ut: checkConsistencyBetweenInterpretedAndCodegen never reaches the backend

Open
#12,860 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### What happens

`checkConsistencyBetweenInterpretedAndCodegen` looks like the strongest assertion in a Gluten expression test, and it never reaches the backend. Example at `gluten-ut/spark35/src/test/scala/org/apache/spark/sql/catalyst/expressions/GlutenMathExpressionsSuite.scala:298`:

```scala
checkConsistencyBetweenInterpretedAndCodegen((e: Expression) => Unhex(e), StringType)
```

`GlutenTestsTrait` overrides `checkEvaluation`, `checkExceptionInExpression` and `checkResult`. It does not override this one. The helper comes from Spark's `ExpressionEvalHelper` and goes through `evaluateWithoutCodegen` plus `GenerateMutableProjection`, never building a DataFrame, so no Gluten plan is produced and Velox is not involved.

### Why it matters

What the line actually compares is vanilla Spark interpreted evaluation against vanilla Spark codegen. If Velox's `unhex` were completely broken, the line would still pass. Because it generates random literals it reads like the broadest check in the case, so it is easy to mistake for a Velox consistency guarantee, and easy to conclude a Gluten expression is well covered when it is not.

This is inherited from Spark's own suites rather than written for Gluten, so it is likely to appear in other `Gluten*ExpressionsSuite` files. Worth a sweep rather than a one-line fix.

### Suggested fix

Either replace it with generated literals driven through `checkEvaluation`, which does go through Gluten, or keep it and add a comment saying it only covers vanilla consistency. The first option is the useful one, provided somebody checks the generated inputs are ones the backend supports.

Found during a review pass on #12840. That PR touched this line only to adapt it to Spark 3.4's two-argument `Unhex`, and left the semantics alone on purpose.

Contributor guide

Open the contributing guide

Research direction

Start with gluten-ut/spark35/src/test/scala/org/apache/spark/sql/catalyst/expressions/GlutenMathExpressionsSuite.scala:298 and inspect GlutenTestsTrait's checkEvaluation, checkExceptionInExpression, and checkResult overrides. Trace ExpressionEvalHelper's checkConsistencyBetweenInterpretedAndCodegen through evaluateWithoutCodegen and GenerateMutableProjection, then sweep other Gluten*ExpressionsSuite files. Done means affected checks either exercise the backend through generated inputs or are explicitly documented as vanilla-only.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
backend, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.