apache / apache/datafusion-python
Attach runtime before calling scan for foreign table provider
- 主要言語
- Python
- スター
- 604
- フォーク
- 174
- 平均マージ
- 1日 7時間
- マージ済み PR(30日)
- 4
説明
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
In the table provider for `lance` in the `scan` method I sometimes have to do a small amount of I/O to load dataset metadata if the metadata is not already in the cache. The `scan` method is async so I assume this is ok. However, I use `spawn_blocking` to trigger this I/O and it fails because the runtime is not active.
**Describe the solution you'd like**
The runtime should be attached before calling `scan`.
**Describe alternatives you've considered**
Currently I work around this by forcing the metadata to be cached before creating the table provider but this is inconvenient.
**Additional context**
Can provide additional details (stack traces, lance reproducers, etc.) on request.
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
調査の方向性
Start at the table provider's async `scan` entry point and the `spawn_blocking` path described in the issue. Reproduce a Lance metadata cache miss, trace when the runtime is attached relative to `scan`, and confirm the cache-miss path completes without the runtime error.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- python
- 領域
- databases
- issue の種類
- 機能追加
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 45/100