apache / apache/arrow-adbc

Implementing drivers in python

Open
#2,292 4 comments 0 reactions 0 assignees View on GitHub
Type: question
Dominant language
C#
Stars
627
Forks
217
Avg merge
17h
Merged PRs (30d)
57

Description

### What would you like help with?

I suppose this is already possible by duck typing classes to look like the ones in `adbc_driver_manager`, but I'm curious what's the general attitude towards implementing new drivers in python. A couple of valid use cases that come to mind are:

- cases when there are already available python packages for the backend that wrap other languages and can output arrow. For example, I eventually opted to go with rust in case of datafusion, but I could have instead implemented it in python using `datafusion-python`. I get that it's not an optimal solution as the end result would only be accessible with python, but it could have accelerated prototype development.
- wrappers around existing drivers that augment underlying drivers with some additional functionality. For example, a python driver that wraps a sqlite driver and adds substrait capabilities by translating substrait to sql before invoking actual commands or a driver that's powered by something like sqlglot and does dialect translations in the python layer.

I'm wondering if it might be a good idea to add a dummy python driver implementation to encourage such use cases.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.