apache / apache/iceberg-python

[feature] Investigate integrations leveraging the PyCapsule protocol

未关闭
#1,655 7 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
1.1k
派生
581
平均合并
1 天 17 小时
30 天内合并 PR
77

描述

### Feature Request / Improvement

Context: https://github.com/apache/iceberg-python/pull/1614#issuecomment-2641089912
Copying the comment over:
Separately, rather than adding more library-specific conversion code, it might make sense for pyiceberg to start leveraging the [PyCapsule protocol](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html) to allow any third party library (dataframe or otherwise) that supports Arrow data to seamlessly consume pyiceberg constructs.

Polars already supports the PyCapsule interface. See https://docs.pola.rs/user-guide/misc/arrow/#using-the-arrow-pycapsule-interface for details.

Implementing the interface on e.g. pyiceberg tables would allow them to be passed directly to dataframe init in polars, just like you can do a pyarrow table today. It also doesn't assume anything about polars support/doesn't add a dependency on polars.

cc @corleyma if you would like to provide more context :)

贡献指南

这个仓库没有索引到贡献指南

调研方向

先了解 PR 1614 的链接评论中的上下文,然后阅读 Arrow PyCapsule 协议和 Polars 的 PyCapsule 接口指南。确定哪些 pyiceberg 表构造应公开该协议,以及如何将它们直接传递给 dataframe 初始化。完成标准是:在不添加 Polars 依赖的情况下,一个有文档说明且经过测试的集成路径能够正常工作。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
data
Issue 类型
功能
难度
4/5
预计耗时
3-5 天
活跃度
冷清
描述清晰度
基本清楚
新手友好度
45/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。