apache / apache/gluten

[Core]NativeColumnToRow is used when disable native scan

Open
#1,299 0 comments 0 reactions 0 assignees View on GitHub
bug stale
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

**Describe the bug**
When native scan is disabled, ( by setting spark.gluten.sql.columnar.filescan = false, for example)
NativeColumnToRow is used instead of ColumnToRow.
```sql
CHNativeColumnarToRow
+- FileScan parquet tpcds_parquet.alltypes_parquet[...
```

**To Reproduce**
set spark.gluten.sql.columnar.filescan =false
select * from parquet_table

**Expected behavior**
Use ColumnToRow when FileSourceScan is vanilla spark node.
```sql
*(1) ColumnarToRow
+- FileScan parquet tpcds_parquet.alltypes_parquet[...
```

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue by disabling spark.gluten.sql.columnar.filescan and running the provided parquet_table query. Inspect the physical plan and the scan-to-row conversion path; done when a vanilla FileSourceScan produces ColumnarToRow rather than NativeColumnarToRow.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.