apache / apache/gluten

[VL][Delta] Gluten Delta stats tracker should fall back when the stats plan can't be offloaded (ClassCastException on TIMESTAMP_NTZ)

Open
#12,538 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

### Backend

VL (Velox)

### Bug description

When Gluten collects Delta write statistics, `GlutenDeltaJobStatsTracker` builds a small local Velox aggregation plan and **assumes the offload always succeeds**, i.e. that the transformed plan is always a `WholeStageTransformer`. When the stats expression references a data type / expression that Velox does not yet support (for example `TIMESTAMP_NTZ`), offload validation correctly rejects the plan and it stays a vanilla Spark `ProjectExec`. The unconditional cast to `WholeStageTransformer` then throws a `ClassCastException`, failing the whole write.

**Expected behavior:** an unsupported type/expression in the Delta stats plan should induce a *fallback* to Spark's native Delta stats collection (the `DeltaJobStatisticsTracker` delegate path), not crash the write.

**Actual behavior:** the write task fails with:

```
java.lang.ClassCastException: class org.apache.spark.sql.execution.ProjectExec cannot be cast
to class org.apache.gluten.execution.WholeStageTransformer
at org.apache.spark.sql.delta.stats.GlutenDeltaJobStatsTracker$GlutenDeltaTaskStatsTracker.(GlutenDeltaJobStatsTracker.scala:174)
at org.apache.spark.sql.delta.stats.GlutenDeltaJobStatsTracker.newTaskInstance(GlutenDeltaJobStatsTracker.scala:80)
at org.apache.spark.sql.execution.datasources.FileFormatDataWriter.$anonfun$statsTrackers$1(FileFormatDataWriter.scala:63)
at scala.collection.immutable.List.map(List.scala:236)
```

### Root cause

In `GlutenDeltaTaskStatsTracker` the transform result is cast unconditionally:

```scala
val veloxTransformer = transformRule(projOp)
val wholeStageTransformer = ColumnarCollapseTransformStages(config)(veloxTransformer)
.asInstanceOf[WholeStageTransformer] // <-- throws when offload was rejected
.child
.asInstanceOf[TransformSupport]
```

`transformRule` runs the offload heuristic with a validator. If the stats expression contains something Velox can't handle (e.g. `TIMESTAMP_NTZ` in an aggregate/function), the validator rejects offload and `transformRule` returns the original vanilla `ProjectExec`. `ColumnarCollapseTransformStages` then leaves it as-is, and `.asInstanceOf[WholeStageTransformer]` fails.

This affects both source variants:
- `backends-velox/src-delta40/main/scala/org/apache/spark/sql/delta/stats/GlutenDeltaJobStatsTracker.scala` (~line 174-177)
- `backends-velox/src-delta33/main/scala/org/apache/spark/sql/delta/stats/GlutenDeltaJobStatsTracker.scala` (~line 170-173)

### How to reproduce

Run the Delta Spark UTs on the Gluten Velox bundle (as in #12388). Any test that collects Delta statistics on a `TIMESTAMP_NTZ` column reproduces it, e.g.:

- `data skipping on TIMESTAMP_NTZ`
- `recompute stats multiple columns and files`

More generally: create/write a Delta table whose statistics columns include a type not yet offloadable by Velox and trigger stats collection.

### Proposed fix

Per the discussion in #12388, rather than fixing `TIMESTAMP_NTZ` support itself (tracked separately in #11622), make the Gluten Delta stats path **detect a rejected/failed offload and fall back** to the native `DeltaJobStatisticsTracker` (or the existing fallback tracker) instead of assuming a `WholeStageTransformer`. Unsupported operations should induce a fallback for Delta plans generally.

### Gluten version

main branch

### Spark version

spark-4.0.x and later (reproduced on the Delta pipeline running Spark 4.1 / Delta v4.2.0; the same unchecked cast exists in the Delta 3.3 source variant).

### References

- PR: #12388 (Delta Spark UT pipeline; this failure is a baseline entry)
- Maintainer discussion: https://github.com/apache/gluten/pull/12388#issuecomment-4979728602 and https://github.com/apache/gluten/pull/12388#issuecomment-4989409743
- Related type-support issue: #11622 ([VL] Support TIMESTAMP_NTZ Type)
- CI run with repro: https://github.com/apache/gluten/actions/runs/29302276900/job/86989997438?pr=12388

_This issue was written with the assistance of AI tooling (GitHub Copilot CLI)._

Contributor guide

Open the contributing guide

Research direction

Start with GlutenDeltaJobStatsTracker.scala in the src-delta40 and src-delta33 variants, focusing on the task tracker construction around the reported lines and the native DeltaJobStatisticsTracker fallback path. Run the Delta Spark UTs referenced in the issue, including the TIMESTAMP_NTZ data-skipping case. Done means rejected stats plans use the native or existing fallback tracker in both variants without a ClassCastException.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.