Make pg_duckdb work with vanilla libduckdb.so
- Dominant language
- C++
- Stars
- 3.2k
- Forks
- 204
- PR merge metrics
- No merged PRs in 30d
Description
### What happens?
I built and installed pg_duckdb, then downloaded the latest binary release, [libduckdb-linux-amd64.zip](https://github.com/duckdb/duckdb/releases/download/v1.1.3/libduckdb-linux-amd64.zip), and put `libduckdb.so` into `/usr/local/lib` so that the OS would find it in the `ld` paths when Postgres requested it. It failed with this error:
```
FATAL: could not load library "/var/lib/postgresql/data/15/lib/pg_duckdb.so": /var/lib/postgresql/data/15/lib/pg_duckdb.so: undefined symbol: _ZNK6duckdb28SimpleNamedParameterFunction8ToStringB5cxx11Ev
```
So I replaced `/usr/local/lib/libduckdb.so` with the file from `pg_config --pkglibdir` (the OS cannot find it there, since it's not in the `ld` paths, so I don't understand why `make install` puts it there). That fixed the issue. Yay!
Except that now the other DuckDB extension I'm working with, alitrack/duckdb_fdw, doesn't work with the `libduckdb.so` compiled by pg_duckdb (and I do have a symlink from `libduckdb.1.1.3.so` so that duckdb_fdw is happy). It complains:
```
ERROR: could not load library "/var/lib/postgresql/data/15/lib/duckdb_fdw.so": /var/lib/postgresql/data/15/lib/duckdb_fdw.so: undefined symbol: _ZN6duckdb9Exception25ConstructMessageRecursiveERKSsRSt6vectorINS_20ExceptionFormatValueESaIS4_EE
```
Maddening.
Would it be possible to get pg_duckdb to work with the binary release of `libduckdb.so`? If not, is there some build configuration to get everything included? It seems like they ought to be, honestly, since [ConstructMessageRecursive](https://github.com/duckdb/duckdb/blob/d0c4cf8a28b062fd41e7fbf5e98c279af12a2ccf/src/common/exception.cpp#L85) has been around for 2 years, and [SimpleNamedParameter](https://github.com/duckdb/duckdb/blob/d0c4cf8a28b062fd41e7fbf5e98c279af12a2ccf/src/function/function.cpp#L59) for four years.
### To Reproduce
``` sh
RUN git clone --depth 1 --branch "v0.2.0" https://github.com/duckdb/pg_duckdb
make -C pg_duckdb install
curl -LO https://github.com/duckdb/duckdb/releases/download/v1.1.3/libduckdb-linux-amd64.zip
unzip -d . libduckdb-linux-amd64.zip libduckdb.so
install -m 755 libduckdb.so /usr/local/lib/libduckdb.so
(cd /usr/local/lib && ln -s libduckdb.so libduckdb.1.1.3.so)
# Add pg_duckdb to shared_preload_libraries and start Postgres
```
### OS:
Linux
### pg_duckdb Version (if built from source use commit hash):
0.2.0
### Postgres Version (if built from source use commit hash):
15.10
### Hardware:
_No response_
### Full Name:
David Wheeler
### Affiliation:
Tembo
### What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.
I have tested with a stable release
### Did you include all relevant data sets for reproducing the issue?
Not applicable - the reproduction does not require a data set
### Did you include all code required to reproduce the issue?
- [x] Yes, I have
### Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?
- [x] Yes, I have
Contributor guide
Research direction
Start by running the supplied reproduction commands for pg_duckdb 0.2.0, then inspect the build and install behavior around make install, pg_config --pkglibdir, and shared-library loading. Done means pg_duckdb loads with the official libduckdb.so while the demonstrated duckdb_fdw setup remains compatible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux, postgresql
- Domain
- build-system, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100