apache / apache/datafusion-python
Remove the need for registering an ObjectStore for remote files
- 主要語言
- Python
- 星號
- 604
- 分支
- 174
- 平均合併
- 1 天 7 小時
- 30 天內合併 PR
- 4
描述
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Currently, we need to register an ObjectStore to register remote files stored in S3, https, etc. This could be more ergonomic from a DX perspective.
**Describe the solution you'd like**
Automatically detect if an ObjectStore is needed, like in the datafusion-cli; see [here](https://github.com/apache/datafusion/blob/6f8c74ca873fe015b2e7ff4eeb2f76bf21ae9d0e/datafusion-cli/src/object_storage.rs#L395).
**Describe alternatives you've considered**
keep it as it is now
**Additional context**
We should increase the test coverage for ObjectStore
貢獻指南
這個儲存庫沒有索引到貢獻指南
研究方向
首先,將 Python ObjectStore 註冊流程與 datafusion-cli/src/object_storage.rs 第395行附近所參照的 datafusion-cli 實作進行比較。追蹤遠端 S3 與 HTTPS 檔案的註冊方式,然後擴充 ObjectStore 的測試涵蓋範圍,並確認在需要的地方能自動偵測到註冊。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python
- 領域
- backend, data-engineering
- Issue 類型
- 功能
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100