apache / apache/datafusion-python

Unable to Read Parquet Files from S3 Bucket

Aberta
#638 0 comentários 0 reações 0 responsáveis Ver no GitHub
bug
Linguagem predominante
Python
Estrelas
604
Forks
174
Merge médio
1d 7h
PRs com merge (30d)
4

Descrição

escription:
I'm attempting to read Parquet files from an S3 bucket using DataFusion in Python. Below is the code snippet I'm using:

python
Copy code
import datafusion
from datafusion import SessionContext

s3 = object_store.object_store.AmazonS3("s3://test/", "us-east-2")
ctx = SessionContext()
ctx.register_object_store("s3", s3)
df = ctx.read_parquet("s3://test/00001.parquet")
Error Encountered:
I'm encountering the following error:

css
Copy code
dataFusion error: Internal("No suitable object store found for s3://test/00001.parquet")
Issue Investigation:
I've tried to find relevant documentation or support resources but haven't been successful in locating any.

Resources Reviewed:
While researching, I came across the following Rust documentation which appears relevant but unfortunately doesn't have a corresponding Python counterpart:

[DataFusion Rust Documentation](https://arrow.apache.org/datafusion-python/generated/datafusion.object_store.object_store.html)
[DataFusion ObjectStore S3 Rust Documentation](https://docs.rs/datafusion-objectstore-s3/latest/datafusion_objectstore_s3/)
Request for Assistance:
Could someone please guide me on how to resolve this issue in Python? Any assistance or pointers to relevant documentation would be greatly appreciated. Thank you!

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Direção de pesquisa

Start with the mentioned SessionContext.register_object_store and read_parquet entry points, then compare the linked Python object-store documentation with the Rust S3 documentation. Reproduce the reported "No suitable object store found" error and determine whether the Python usage is supported or needs documentation or a tested correction.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
aws, python
Domínio
cloud
Tipo de issue
Bug
Dificuldade
3/5
Tempo estimado
1-2 dias
Status de atividade
Estagnada
Clareza
Precisa de esclarecimento
Facilidade para iniciantes
30/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.