apache / apache/datafusion-python
aws.XXX options are not recognized in SQL API for CREATE EXTERNAL TABLE with AWS S3 location
- Lenguaje dominante
- Python
- Estrellas
- 604
- Forks
- 174
- Merge medio
- 1 d 7 h
- PR fusionados (30 d)
- 4
Descripción
**Describe the bug**
Hi
I was trying to create external table from AWS S3 using examples from [Datafusion CLI docs](https://datafusion.apache.org/user-guide/cli/datasources.html#s3) and [SQL DDL docs](https://datafusion.apache.org/user-guide/sql/ddl.html#create-external-table), see **To Reproduce** section below. But looks like AWS-specific options for external tables doesn't work with SQL API in the same way as it described in CLI docs
```Exception: DataFusion error: Configuration("Could not find config namespace \"aws\"")```
I can successfully register and read the same parquet with the same credentials via register_object_store + register_parquet functions, so there's no problem with the data or S3 connection
**To Reproduce**
```python
import datafusion
ctx = datafusion.SessionContext()
raw_data_df = ctx.sql(f"""
create external table raw_data
stored as parquet
options
(
'aws.access_key_id' '***',
'aws.secret_access_key' '***',
'aws.region' 'eu-central-1'
)
location 's3://some-bucket/some-folder/'
""")
raw_data_df.show()
```
**Expected behavior**
```create external table``` command works with ```aws.xxx``` options
**Additional context**
v 42.0.0
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.