apache / apache/datafusion-python
Pipeline FFI types: missing Protocols, ABCs, and capsule helpers
- 主要語言
- Python
- 星號
- 604
- 分支
- 174
- 平均合併
- 1 天 7 小時
- 30 天內合併 PR
- 4
描述
## Background
A v54 upstream coverage audit identified six FFI-pipeline gaps where DataFusion's FFI types are already imported on the Rust side but the Python surface is incomplete -- either missing a typed `Protocol` / `ABC` for users to implement against, or missing a `from_pycapsule` helper that the rest of the FFI surface uses. These are not v54-specific; they predate the v54 release. Filing this umbrella issue so the gaps are tracked together rather than disappearing into the audit report.
## Items
1. **`FFI_TableFunction`** -- no `TableFunctionExportable(Protocol)`. The duck-typed `hasattr` check at `user_defined.py:1161,1205` should be replaced by a typed protocol mirroring `TableProviderExportable`.
2. **`FFI_TableProvider`** -- no Python `TableProvider(ABC)` analogous to `CatalogProvider` / `SchemaProvider`. Users implementing a custom table provider in Python have no abstract base class to subclass; the Exportable protocol is the only entry point.
3. **`FFI_ExtensionOptions`** -- no `ExtensionOptionsExportable(Protocol)`. Currently consumed only at the `SessionConfig.with_extension` call site without a typed protocol describing the expected capsule export.
4. **`FFI_TaskContextProvider`** -- no Python `Protocol` and no example. Producer-only at `PySessionContext.__datafusion_task_context_provider__`; nothing imports one.
5. **`FFI_TableProviderFactory`** -- no `from_pycapsule` helper on `RustWrappedPyTableProviderFactory`. The capsule decode is inlined at `context.rs:728-749` rather than going through the `from_pycapsule!` macro the other FFI capsule importers use.
6. **`WindowUDF`** -- no Python `ABC` equivalent to `Accumulator` for native Python window UDFs. Users can register a Rust-defined window UDF over FFI but cannot define one in pure Python.
## Why deferred
Each item is small in isolation but they form a self-contained cleanup track. The audit grouped them so the work can be picked up together by a contributor focused on FFI ergonomics, rather than scattered across the v54 gap-closure PRs which focus on net-new surface area. Picking up any one item from this list is welcome.
貢獻指南
這個儲存庫沒有索引到貢獻指南
研究方向
首先,將現有的 TableProviderExportable、CatalogProvider、SchemaProvider、Accumulator 和 from_pycapsule! 實作與 user_defined.py、context.rs、SessionConfig.with_extension 和 PySessionContext.__datafusion_task_context_provider__ 中命名的入口進行比較。追蹤列出的每個 FFI 型別及其現有的 Rust import 或 capsule 路徑。六個缺少的具型別介面、helper 和範例都得到一致涵蓋,且相關測試通過後,即視為完成。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python, rust
- 領域
- api, developer-experience
- Issue 類型
- 功能
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 冷清
- 描述清晰度
- 基本清楚
- 新手友好度
- 48/100