apache / apache/datafusion

`schema_force_view_type` configuration not working for `CREATE EXTERNAL TABLE`

Aperta
#14,909 4 commenti 0 reazioni 1 assegnatario Rivendicata da @zhuqi-lucas Vedi su GitHub
bug
Lingua principale
Rust
Stelle
9.3k
Fork
2.4k
Merge medio
3g 11h
PR unite (30g)
360

Descrizione

### Describe the bug

`schema_force_view_type` configuration said:
```
datafusion.execution.parquet.schema_force_view_types | true | (reading) If true, parquet reader will read columns of Utf8/Utf8Large with Utf8View, and Binary/BinaryLarge with BinaryView.
```
Reference: https://datafusion.apache.org/user-guide/configs.html

I tried the following script within `datafusion-cli`
```sql
set datafusion.execution.parquet.schema_force_view_types = true;

CREATE EXTERNAL TABLE IF NOT EXISTS lineitem (
l_orderkey BIGINT,
l_partkey BIGINT,
l_suppkey BIGINT,
l_linenumber INTEGER,
l_quantity DECIMAL(15, 2),
l_extendedprice DECIMAL(15, 2),
l_discount DECIMAL(15, 2),
l_tax DECIMAL(15, 2),
l_returnflag VARCHAR,
l_linestatus VARCHAR,
l_shipdate DATE,
l_commitdate DATE,
l_receiptdate DATE,
l_shipinstruct VARCHAR,
l_shipmode VARCHAR,
l_comment VARCHAR
) STORED AS parquet
LOCATION '/Users/yongting/Code/datafusion/benchmarks/data/tpch_sf10/lineitem';

select arrow_typeof(l_comment) from lineitem limit 1;
```

The data type is `Utf8` instead `Utf8View`
```
+----------------------------------+
| arrow_typeof(lineitem.l_comment) |
+----------------------------------+
| Utf8 |
+----------------------------------+
```

### To Reproduce

_No response_

### Expected behavior

_No response_

### Additional context

_No response_

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.