apache / apache/iceberg-python
Pluggable Scan Planning
- 主要語言
- Python
- 星號
- 1.1k
- 分支
- 581
- 平均合併
- 1 天 13 小時
- 30 天內合併 PR
- 76
描述
### Feature Request / Improvement
Umbrella issue. Today we have tightly coupled query planning in Python. This works well but runs into limitations when it comes to concurrency and performance. Having pluggable query planning will allow for:
- **Python based query planning** which is working today and is proven.
- **Iceberg-Rust based query planning** as suggeted in https://github.com/apache/iceberg-python/issues/2159#issuecomment-3033339213.
- **REST Catalog server side planning** this leverages the REST Catalog to plan the query, rather than the client. First issue can be found here: https://github.com/apache/iceberg-python/issues/2302.
As an API, I think it makes sense to follow the Server Side planning. So a `PlanTableScanRequest` as an input, and the [ScanTasks](https://github.com/apache/iceberg/blob/68a241615e104d97a86c932c381d7dfac802d769/open-api/rest-catalog-open-api.py#L1290) as the output.
貢獻指南
這個儲存庫沒有索引到貢獻指南
研究方向
先閱讀這個總 issue,以及 issue 2302;後者涵蓋 REST Catalog 首項伺服器端規劃工作。檢視提議的 PlanTableScanRequest 輸入和 ScanTasks 輸出,然後將它們與現有的 Python 查詢規劃路徑進行比較。完成的標準是達成一個獲得共識的可插拔規劃 API,能夠支援 Python、Iceberg-Rust 和 REST Catalog 規劃。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python, rust
- 領域
- backend-api-design, distributed-systems
- Issue 類型
- 功能
- 難度
- 5/5
- 預估耗時
- 一週以上
- 活躍度
- 冷清
- 描述清晰度
- 需要釐清
- 新手友好度
- 25/100