apache / apache/datafusion-python
Attach runtime before calling scan for foreign table provider
- 主要語言
- Python
- 星號
- 604
- 分支
- 174
- 平均合併
- 1 天 7 小時
- 30 天內合併 PR
- 4
描述
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
In the table provider for `lance` in the `scan` method I sometimes have to do a small amount of I/O to load dataset metadata if the metadata is not already in the cache. The `scan` method is async so I assume this is ok. However, I use `spawn_blocking` to trigger this I/O and it fails because the runtime is not active.
**Describe the solution you'd like**
The runtime should be attached before calling `scan`.
**Describe alternatives you've considered**
Currently I work around this by forcing the metadata to be cached before creating the table provider but this is inconvenient.
**Additional context**
Can provide additional details (stack traces, lance reproducers, etc.) on request.
貢獻指南
這個儲存庫沒有索引到貢獻指南
研究方向
從表格提供者的非同步 `scan` 進入點以及 issue 中描述的 `spawn_blocking` 路徑開始。重現一次 Lance 中繼資料快取未命中,追蹤 runtime 相對於 `scan` 的附加時機,並確認快取未命中路徑能在不發生 runtime 錯誤的情況下完成。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python
- 領域
- databases
- Issue 類型
- 功能
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 基本清楚
- 新手友好度
- 45/100