[VL][Delta] Gluten Delta stats tracker should fall back when the stats plan can't be offloaded (ClassCastException on TIMESTAMP_NTZ)
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 85
Description
### Backend
VL (Velox)
### Bug description
When Gluten collects Delta write statistics, `GlutenDeltaJobStatsTracker` builds a small local Velox aggregation plan and **assumes the offload always succeeds**, i.e. that the transformed plan is always a `WholeStageTransformer`. When the stats expression references a data type / expression that Velox does not yet support (for example `TIMESTAMP_NTZ`), offload validation correctly rejects the plan and it stays a vanilla Spark `ProjectExec`. The unconditional cast to `WholeStageTransformer` then throws a `ClassCastException`, failing the whole write.
**Expected behavior:** an unsupported type/expression in the Delta stats plan should induce a *fallback* to Spark's native Delta stats collection (the `DeltaJobStatisticsTracker` delegate path), not crash the write.
**Actual behavior:** the write task fails with:
```
java.lang.ClassCastException: class org.apache.spark.sql.execution.ProjectExec cannot be cast
to class org.apache.gluten.execution.WholeStageTransformer
at org.apache.spark.sql.delta.stats.GlutenDeltaJobStatsTracker$GlutenDeltaTaskStatsTracker.(GlutenDeltaJobStatsTracker.scala:174)
at org.apache.spark.sql.delta.stats.GlutenDeltaJobStatsTracker.newTaskInstance(GlutenDeltaJobStatsTracker.scala:80)
at org.apache.spark.sql.execution.datasources.FileFormatDataWriter.$anonfun$statsTrackers$1(FileFormatDataWriter.scala:63)
at scala.collection.immutable.List.map(List.scala:236)
```
### Root cause
In `GlutenDeltaTaskStatsTracker` the transform result is cast unconditionally:
```scala
val veloxTransformer = transformRule(projOp)
val wholeStageTransformer = ColumnarCollapseTransformStages(config)(veloxTransformer)
.asInstanceOf[WholeStageTransformer] // <-- throws when offload was rejected
.child
.asInstanceOf[TransformSupport]
```
`transformRule` runs the offload heuristic with a validator. If the stats expression contains something Velox can't handle (e.g. `TIMESTAMP_NTZ` in an aggregate/function), the validator rejects offload and `transformRule` returns the original vanilla `ProjectExec`. `ColumnarCollapseTransformStages` then leaves it as-is, and `.asInstanceOf[WholeStageTransformer]` fails.
This affects both source variants:
- `backends-velox/src-delta40/main/scala/org/apache/spark/sql/delta/stats/GlutenDeltaJobStatsTracker.scala` (~line 174-177)
- `backends-velox/src-delta33/main/scala/org/apache/spark/sql/delta/stats/GlutenDeltaJobStatsTracker.scala` (~line 170-173)
### How to reproduce
Run the Delta Spark UTs on the Gluten Velox bundle (as in #12388). Any test that collects Delta statistics on a `TIMESTAMP_NTZ` column reproduces it, e.g.:
- `data skipping on TIMESTAMP_NTZ`
- `recompute stats multiple columns and files`
More generally: create/write a Delta table whose statistics columns include a type not yet offloadable by Velox and trigger stats collection.
### Proposed fix
Per the discussion in #12388, rather than fixing `TIMESTAMP_NTZ` support itself (tracked separately in #11622), make the Gluten Delta stats path **detect a rejected/failed offload and fall back** to the native `DeltaJobStatisticsTracker` (or the existing fallback tracker) instead of assuming a `WholeStageTransformer`. Unsupported operations should induce a fallback for Delta plans generally.
### Gluten version
main branch
### Spark version
spark-4.0.x and later (reproduced on the Delta pipeline running Spark 4.1 / Delta v4.2.0; the same unchecked cast exists in the Delta 3.3 source variant).
### References
- PR: #12388 (Delta Spark UT pipeline; this failure is a baseline entry)
- Maintainer discussion: https://github.com/apache/gluten/pull/12388#issuecomment-4979728602 and https://github.com/apache/gluten/pull/12388#issuecomment-4989409743
- Related type-support issue: #11622 ([VL] Support TIMESTAMP_NTZ Type)
- CI run with repro: https://github.com/apache/gluten/actions/runs/29302276900/job/86989997438?pr=12388
_This issue was written with the assistance of AI tooling (GitHub Copilot CLI)._
Contributor guide
Research direction
Start with GlutenDeltaJobStatsTracker.scala in the src-delta40 and src-delta33 variants, focusing on the task tracker construction around the reported lines and the native DeltaJobStatisticsTracker fallback path. Run the Delta Spark UTs referenced in the issue, including the TIMESTAMP_NTZ data-skipping case. Done means rejected stats plans use the native or existing fallback tracker in both variants without a ClassCastException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100