duckdb / duckdb/pg_duckdb

Make pg_duckdb work with vanilla libduckdb.so

Open
#554 7 comments 0 reactions 0 assignees View on GitHub
packaging
Dominant language
C++
Stars
3.2k
Forks
204
PR merge metrics
No merged PRs in 30d

Description

### What happens?

I built and installed pg_duckdb, then downloaded the latest binary release, [libduckdb-linux-amd64.zip](https://github.com/duckdb/duckdb/releases/download/v1.1.3/libduckdb-linux-amd64.zip), and put `libduckdb.so` into `/usr/local/lib` so that the OS would find it in the `ld` paths when Postgres requested it. It failed with this error:

```
FATAL: could not load library "/var/lib/postgresql/data/15/lib/pg_duckdb.so": /var/lib/postgresql/data/15/lib/pg_duckdb.so: undefined symbol: _ZNK6duckdb28SimpleNamedParameterFunction8ToStringB5cxx11Ev
```

So I replaced `/usr/local/lib/libduckdb.so` with the file from `pg_config --pkglibdir` (the OS cannot find it there, since it's not in the `ld` paths, so I don't understand why `make install` puts it there). That fixed the issue. Yay!

Except that now the other DuckDB extension I'm working with, alitrack/duckdb_fdw, doesn't work with the `libduckdb.so` compiled by pg_duckdb (and I do have a symlink from `libduckdb.1.1.3.so` so that duckdb_fdw is happy). It complains:

```
ERROR: could not load library "/var/lib/postgresql/data/15/lib/duckdb_fdw.so": /var/lib/postgresql/data/15/lib/duckdb_fdw.so: undefined symbol: _ZN6duckdb9Exception25ConstructMessageRecursiveERKSsRSt6vectorINS_20ExceptionFormatValueESaIS4_EE
```

Maddening.

Would it be possible to get pg_duckdb to work with the binary release of `libduckdb.so`? If not, is there some build configuration to get everything included? It seems like they ought to be, honestly, since [ConstructMessageRecursive](https://github.com/duckdb/duckdb/blob/d0c4cf8a28b062fd41e7fbf5e98c279af12a2ccf/src/common/exception.cpp#L85) has been around for 2 years, and [SimpleNamedParameter](https://github.com/duckdb/duckdb/blob/d0c4cf8a28b062fd41e7fbf5e98c279af12a2ccf/src/function/function.cpp#L59) for four years.

### To Reproduce

``` sh
RUN git clone --depth 1 --branch "v0.2.0" https://github.com/duckdb/pg_duckdb
make -C pg_duckdb install
curl -LO https://github.com/duckdb/duckdb/releases/download/v1.1.3/libduckdb-linux-amd64.zip
unzip -d . libduckdb-linux-amd64.zip libduckdb.so
install -m 755 libduckdb.so /usr/local/lib/libduckdb.so
(cd /usr/local/lib && ln -s libduckdb.so libduckdb.1.1.3.so)
# Add pg_duckdb to shared_preload_libraries and start Postgres
```

### OS:

Linux

### pg_duckdb Version (if built from source use commit hash):

0.2.0

### Postgres Version (if built from source use commit hash):

15.10

### Hardware:

_No response_

### Full Name:

David Wheeler

### Affiliation:

Tembo

### What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

### Did you include all relevant data sets for reproducing the issue?

Not applicable - the reproduction does not require a data set

### Did you include all code required to reproduce the issue?

- [x] Yes, I have

### Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?

- [x] Yes, I have

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied reproduction commands for pg_duckdb 0.2.0, then inspect the build and install behavior around make install, pg_config --pkglibdir, and shared-library loading. Done means pg_duckdb loads with the official libduckdb.so while the demonstrated duckdb_fdw setup remains compatible.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, linux, postgresql
Domain
build-system, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.