apache / apache/datafusion-python

aws.XXX options are not recognized in SQL API for CREATE EXTERNAL TABLE with AWS S3 location

Aberta
#970 0 comentários 0 reações 0 responsáveis Ver no GitHub
bug
Linguagem predominante
Python
Estrelas
604
Forks
174
Merge médio
1d 7h
PRs com merge (30d)
4

Descrição

**Describe the bug**
Hi

I was trying to create external table from AWS S3 using examples from [Datafusion CLI docs](https://datafusion.apache.org/user-guide/cli/datasources.html#s3) and [SQL DDL docs](https://datafusion.apache.org/user-guide/sql/ddl.html#create-external-table), see **To Reproduce** section below. But looks like AWS-specific options for external tables doesn't work with SQL API in the same way as it described in CLI docs

```Exception: DataFusion error: Configuration("Could not find config namespace \"aws\"")```

I can successfully register and read the same parquet with the same credentials via register_object_store + register_parquet functions, so there's no problem with the data or S3 connection

**To Reproduce**
```python
import datafusion

ctx = datafusion.SessionContext()

raw_data_df = ctx.sql(f"""
create external table raw_data
stored as parquet
options
(
'aws.access_key_id' '***',
'aws.secret_access_key' '***',
'aws.region' 'eu-central-1'
)
location 's3://some-bucket/some-folder/'
""")

raw_data_df.show()
```

**Expected behavior**
```create external table``` command works with ```aws.xxx``` options

**Additional context**
v 42.0.0

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.