apache / apache/datafusion-python

Attach runtime before calling scan for foreign table provider

Offen
#1,146 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
enhancement
Vorherrschende Sprache
Python
Sterne
604
Forks
174
Ø Merge
1 T. 7 Std.
Gemergte PRs (30 T.)
4

Beschreibung

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

In the table provider for `lance` in the `scan` method I sometimes have to do a small amount of I/O to load dataset metadata if the metadata is not already in the cache. The `scan` method is async so I assume this is ok. However, I use `spawn_blocking` to trigger this I/O and it fails because the runtime is not active.

**Describe the solution you'd like**

The runtime should be attached before calling `scan`.

**Describe alternatives you've considered**

Currently I work around this by forcing the metadata to be cached before creating the table provider but this is inconvenient.

**Additional context**

Can provide additional details (stack traces, lance reproducers, etc.) on request.

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.