apache / apache/datafusion-python

Attach runtime before calling scan for foreign table provider

未关闭
#1,146 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
enhancement
主要语言
Python
星标
604
派生
174
平均合并
1 天 7 小时
30 天内合并 PR
4

描述

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

In the table provider for `lance` in the `scan` method I sometimes have to do a small amount of I/O to load dataset metadata if the metadata is not already in the cache. The `scan` method is async so I assume this is ok. However, I use `spawn_blocking` to trigger this I/O and it fails because the runtime is not active.

**Describe the solution you'd like**

The runtime should be attached before calling `scan`.

**Describe alternatives you've considered**

Currently I work around this by forcing the metadata to be cached before creating the table provider but this is inconvenient.

**Additional context**

Can provide additional details (stack traces, lance reproducers, etc.) on request.

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。