[VL][Ubuntu] Build fails linking libvelox.so: system libgflags.a not compiled with -fPIC
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 85
Description
## Environment
- **OS**: Ubuntu 22.04 (also affects Ubuntu 20.04)
- **Backend**: Velox
- **Build type**: Dynamic (non-vcpkg)
## Problem
Building Gluten with the Velox backend on Ubuntu fails at the `libvelox.so` link step with:
```
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libgflags.a(gflags.cc.o): relocation R_X86_64_PC32 against symbol 'stderr@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
```
## Root Cause
`scripts/setup-ubuntu.sh` installs gflags via `apt-get install libgflags-dev`. The apt-packaged `libgflags.a` on Ubuntu is **not compiled with `-fPIC`**, so the linker rejects it when it is pulled into `libvelox.so` (a shared library).
The issue is in `setup-ubuntu.sh:113`:
```bash
libgflags-dev \ # <-- apt version is not PIC-compiled
```
Folly's installed cmake config (`/usr/local/lib/cmake/folly/folly-targets.cmake`) hardcodes `gflags_static` as a link dependency, which resolves to the non-PIC `/usr/lib/x86_64-linux-gnu/libgflags.a`.
## Why Only Ubuntu
`setup-centos8.sh` and `setup-openeuler24.sh` both have a dedicated `install_gflags` function that:
1. Removes the system package (`dnf remove -y gflags`)
2. Builds gflags from source with `cmake_install_dir gflags -DBUILD_SHARED_LIBS=ON -DBUILD_STATIC_LIBS=ON`
`setup-ubuntu.sh` is missing this function entirely.
## Fix
Add an `install_gflags` function to `setup-ubuntu.sh`, mirroring what `setup-centos8.sh` does:
```bash
function install_gflags {
# Remove apt version (not PIC-compiled, cannot be linked into shared libs)
sudo apt-get remove -y libgflags-dev
wget_and_untar https://github.com/gflags/gflags/archive/v2.2.2.tar.gz gflags
cmake_install_dir gflags \
-DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_STATIC_LIBS=ON \
-DBUILD_gflags_LIB=ON
}
```
And call it from `install_dependencies` before building Folly (since Folly's cmake export will then reference the PIC-compiled version).
## Workaround
Until fixed, users on Ubuntu can manually build gflags with PIC before running the Gluten build:
```bash
sudo apt-get remove -y libgflags-dev
cd /tmp
wget https://github.com/gflags/gflags/archive/v2.2.2.tar.gz -O gflags.tar.gz
tar -xzf gflags.tar.gz && cd gflags-2.2.2
cmake -B build \
-DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_STATIC_LIBS=ON \
-DBUILD_gflags_LIB=ON \
-DCMAKE_INSTALL_PREFIX=/usr/local \
-DCMAKE_BUILD_TYPE=Release
sudo cmake --build build -j4 --target install
# Reinstall glog-dev (removed as a side effect)
sudo apt-get install -y libgoogle-glog-dev
```
## Note
Removing `libgflags-dev` also removes `libgoogle-glog-dev` as a side effect (apt dependency), causing a secondary `glog/logging.h: No such file or directory` error. The workaround above includes reinstalling `libgoogle-glog-dev`.
Contributor guide
Research direction
Start in scripts/setup-ubuntu.sh at line 113 and compare its dependency flow with install_gflags in setup-centos8.sh and setup-openeuler24.sh. Add the Ubuntu gflags installation step before Folly is built, then run the dynamic Velox build on Ubuntu and confirm libvelox.so links without the non-PIC libgflags.a error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, cmake, ubuntu
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100