apache / apache/iceberg-python

[feature] Investigate integrations leveraging the PyCapsule protocol

未關閉
#1,655 7 則留言 1 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
1.1k
分支
581
平均合併
1 天 17 小時
30 天內合併 PR
78

描述

### Feature Request / Improvement

Context: https://github.com/apache/iceberg-python/pull/1614#issuecomment-2641089912
Copying the comment over:
Separately, rather than adding more library-specific conversion code, it might make sense for pyiceberg to start leveraging the [PyCapsule protocol](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html) to allow any third party library (dataframe or otherwise) that supports Arrow data to seamlessly consume pyiceberg constructs.

Polars already supports the PyCapsule interface. See https://docs.pola.rs/user-guide/misc/arrow/#using-the-arrow-pycapsule-interface for details.

Implementing the interface on e.g. pyiceberg tables would allow them to be passed directly to dataframe init in polars, just like you can do a pyarrow table today. It also doesn't assume anything about polars support/doesn't add a dependency on polars.

cc @corleyma if you would like to provide more context :)

貢獻指南

這個儲存庫沒有索引到貢獻指南

研究方向

先了解 PR 1614 的連結留言中的上下文,然後閱讀 Arrow PyCapsule 協定和 Polars 的 PyCapsule 介面指南。確認哪些 pyiceberg 資料表建構應公開該協定,以及如何將它們直接傳遞給 dataframe 初始化。完成標準是:在不新增 Polars 相依性的情況下,一個有文件說明且經過測試的整合路徑能夠正常運作。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
data
Issue 類型
功能
難度
4/5
預估耗時
3-5 天
活躍度
冷清
描述清晰度
基本清楚
新手友好度
45/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。