apache / apache/datafusion-python

Remove the need for registering an ObjectStore for remote files

Open
#899 16 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
604
Forks
174
Avg merge
1d 7h
Merged PRs (30d)
4

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Currently, we need to register an ObjectStore to register remote files stored in S3, https, etc. This could be more ergonomic from a DX perspective.

**Describe the solution you'd like**
Automatically detect if an ObjectStore is needed, like in the datafusion-cli; see [here](https://github.com/apache/datafusion/blob/6f8c74ca873fe015b2e7ff4eeb2f76bf21ae9d0e/datafusion-cli/src/object_storage.rs#L395).

**Describe alternatives you've considered**
keep it as it is now

**Additional context**
We should increase the test coverage for ObjectStore

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.