apache / apache/gluten

[CH] Columnar to row error when plan has InMemoryTableScanExec

Open
#829 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

**Describe the bug**
spark plan
```text
Project [array_repeat(a#2, 2) AS array_repeat(a, 2)#16]
+- ColumnarToRow
+- InMemoryTableScan [a#2]
+- InMemoryRelation [a#2, b#3], StorageLevel(disk, memory, deserialized, 1 replicas)
+- *(1) Scan ExistingRDD[a#2,b#3] Project [array_repeat(a#2, 2) AS array_repeat(a, 2)#16]
```
In ch backend
ColumnarToRow will be transform to `BlockGlutenColumnarToRowExec`, `InMemoryTableScan` return spark columnar data, it makes error in `CHNativeBlock.fromColumnarBatch(batch)`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.