apache / apache/arrow-adbc

docs: document driver binary and entrypoint naming convention

Open
#4,751 3 comments 0 reactions 0 assignees View on GitHub
Type: bug Type: enhancement
Dominant language
C#
Stars
627
Forks
217
Avg merge
17h
Merged PRs (30d)
57

Description

### What feature or improvement would you like to see?

When I first set up `adbc_clickhouse`, I didn't realize that there was an expected naming convention for ADBC drivers: https://github.com/ClickHouse/adbc_clickhouse/pull/66

As of writing, this is not really documented anywhere:

* https://arrow.apache.org/adbc/24/driver/authoring.html
* https://arrow.apache.org/adbc/24/format/driver_manifests.html

The only discussion I found regarding naming conventions in my initial research was in the docs for `adbc_ffi::export_driver!()`, which doesn't prescribe a naming convention but just explains how the entrypoint symbol lookup works: https://docs.rs/adbc_ffi/0.23.0/adbc_ffi/macro.export_driver.html

I chose `adbc_clickhouse` (and `AdbcClickhouseInit`) because it felt more concise than `adbc_driver_clickhouse` (and `AdbcDriverClickhouseInit`) while still seeming to work for the rules laid out by `export_driver!()`. It loaded just fine with the Rust `adbc_driver_manager` crate.

The example dummy driver for Rust appears to support this choice:

* https://github.com/apache/arrow-adbc/blob/f1d6412b809784a882ad1c971018e4401c91aecd/rust/driver/dummy/Cargo.toml#L19
* https://github.com/apache/arrow-adbc/blob/f1d6412b809784a882ad1c971018e4401c91aecd/rust/driver/dummy/src/lib.rs#L941

However, it sounds like some driver managers _specifically_ look for `AdbcDriver*Init` and break if you try to load a driver that doesn't follow that convention. Unfortunately, https://github.com/ClickHouse/adbc_clickhouse/pull/66 is all the context I have here.

To work around this, the driver binary ends up getting renamed during the release process to `libadbc_driver_clickhouse.so`: https://github.com/adbc-drivers/clickhouse/blob/main/src/ci/scripts/build.sh#L46

And an additional `AdbcDriverClickhouseInit` entrypoint gets added for compatibility with old driver managers that don't recognize the `entrypoint` manifest key: https://github.com/adbc-drivers/clickhouse/blob/41ef9369dae4c594edc926cdf6858140a949cdb3/src/ci/scripts/pre-build.sh#L33-L39

I'm not really comfortable with this situation because it means the driver installed through `dbc` has a different filename and entrypoint than if the driver is built manually, which could confuse and frustrate users trying to contribute to the driver or test the latest changes on `main`.

We're prepared to resolve this for good on our end (https://github.com/ClickHouse/adbc_clickhouse/issues/75). However, this could have been avoided if the expected naming conventions were documented better.

Contributor guide

Open the contributing guide

Research direction

Start with the ADBC driver authoring and driver manifest documentation, then compare the adbc_ffi::export_driver!() docs and Rust dummy driver examples. Review the linked ClickHouse build.sh and pre-build.sh workarounds to identify the naming and entrypoint compatibility rules. Done means the expected convention and compatibility behavior are documented clearly enough for manually built and released drivers.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.