apache / apache/gluten

[VL] Fallback summary not populated if InMemoryTableScan is present in spark plan

Open
#8,488 0 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

When plan contains InMemoryTableScan, if in its subplan if there is any node which fallbacks to spark, then that information is not available in Spark UI. @ulysses-you @JkSelf Is this a known issue?
```
== Fallback Summary ==
(1) InMemoryTableScan: Columnar table cache is disabled

== Physical Plan ==
Execute InsertIntoHadoopFsRelationCommand (40)
+- VeloxColumnarWriteFiles (39)
:- ^ WriteFilesExecTransformer (34)
: +- ^ ProjectExecTransformer (33)
: +- ^ InputIteratorTransformer (32)
: +- ShuffleQueryStage (30), Statistics(sizeInBytes=134.0 MiB, rowCount=2.42E+5)
: +- ColumnarExchange (29)
: +- VeloxResizeBatches (28)
: +- ^ ProjectExecTransformer (26)
: +- ^ SortExecTransformer (25)
: +- ^ ProjectExecTransformer (24)
: +- ^ InputIteratorTransformer (23)
: +- RowToVeloxColumnar (21)
: +- TableCacheQueryStage (20), Statistics(sizeInBytes=133.7 MiB, rowCount=2.42E+5)
: +- InMemoryTableScan (1)
: +- InMemoryRelation (2)
: +- AdaptiveSparkPlan (19)
: +- Project (18)
: +- BroadcastHashJoin LeftOuter BuildRight (17)
: :- Project (13)
: : +- Project (12)
: : +- Filter (11)
: : +- Window (10)
: : +- WindowGroupLimit (9)
: : +- Sort (8)
: : +- Exchange (7)
: : +- WindowGroupLimit (6)
: : +- Sort (5)
: : +- Project (4)
: : +- Scan hive spark_catalog.raw_www (3)
: +- BroadcastExchange (16)
: +- Filter (15)
: +- Scan hive spark_catalog.stg_www (14)
+- RowToVeloxColumnar (38)
+- WriteFiles (37)
+- NoopLeaf (36)
```

### Spark version

Spark-3.5.x

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.