apache / apache/datafusion-python
Attach runtime before calling scan for foreign table provider
- Vorherrschende Sprache
- Python
- Sterne
- 604
- Forks
- 174
- Ø Merge
- 1 T. 7 Std.
- Gemergte PRs (30 T.)
- 4
Beschreibung
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
In the table provider for `lance` in the `scan` method I sometimes have to do a small amount of I/O to load dataset metadata if the metadata is not already in the cache. The `scan` method is async so I assume this is ok. However, I use `spawn_blocking` to trigger this I/O and it fails because the runtime is not active.
**Describe the solution you'd like**
The runtime should be attached before calling `scan`.
**Describe alternatives you've considered**
Currently I work around this by forcing the metadata to be cached before creating the table provider but this is inconvenient.
**Additional context**
Can provide additional details (stack traces, lance reproducers, etc.) on request.
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Bewertung
Dieses Issue wurde noch nicht bewertet.