apache / apache/arrow-nanoarrow

Tests fail linking against libarrow.so built in Conda environment

Open
#254 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
249
Forks
68
Avg merge
2d 21h
Merged PRs (30d)
11

Description

I have a Conda environment where I do Arrow development.
When I try to build nanoarrow in that same environment I get linking errors in tests.
```
[23/25] Linking CXX executable utils_test
FAILED: utils_test utils_test[1]_tests.cmake /home/antoine/arrow/nanoarrow/build-debug/utils_test[1]_tests.cmake
: && /home/antoine/mambaforge/envs/pyarrow/bin/g++ -g CMakeFiles/utils_test.dir/src/nanoarrow/utils_test.cc.o -o utils_test -Wl,-rpath,/home/antoine/mambaforge/envs/pyarrow/lib libnanoarrow.a lib/libgtest_maind.a /home/antoine/mambaforge/envs/pyarrow/lib/libarrow.so.1300.0.0 lib/libgtestd.a -lpthread && cd /home/antoine/arrow/nanoarrow/build-debug && /home/antoine/mambaforge/envs/pyarrow/bin/cmake -D TEST_TARGET=utils_test -D TEST_EXECUTABLE=/home/antoine/arrow/nanoarrow/build-debug/utils_test -D TEST_EXECUTOR= -D TEST_WORKING_DIR=/home/antoine/arrow/nanoarrow/build-debug -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=utils_test_TESTS -D CTEST_FILE=/home/antoine/arrow/nanoarrow/build-debug/utils_test[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=5 -D TEST_XML_OUTPUT_DIR= -P /home/antoine/mambaforge/envs/pyarrow/share/cmake-3.26/Modules/GoogleTestAddTests.cmake
/home/antoine/mambaforge/envs/pyarrow/bin/../lib/gcc/x86_64-conda-linux-gnu/13.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: libcrypto.so.3, needed by /home/antoine/mambaforge/envs/pyarrow/lib/libarrow.so.1300.0.0, not found (try using -rpath or -rpath-link)
[...]
/home/antoine/mambaforge/envs/pyarrow/bin/../lib/gcc/x86_64-conda-linux-gnu/13.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/antoine/mambaforge/envs/pyarrow/lib/libarrow.so.1300.0.0: undefined reference to `BrotliEncoderCompressStream'
[etc.]
```

nanoarrow is configured using:
```
cmake .. -DCMAKE_BUILD_RPATH=${CONDA_PREFIX}/lib -DCMAKE_BUILD_TYPE=debug -DNANOARROW_BUILD_TESTS=on -GNinja
```

Note that all libraries do resolve from the command line:
```
$ lddtree /home/antoine/mambaforge/envs/pyarrow/lib/libarrow.so.1300.0.0
libarrow.so.1300.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libarrow.so.1300.0.0 (interpreter => none)
libcrypto.so.3 => /home/antoine/mambaforge/envs/pyarrow/lib/libcrypto.so.3
libssl.so.3 => /home/antoine/mambaforge/envs/pyarrow/lib/libssl.so.3
libbrotlienc.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libbrotlienc.so.1
libbrotlidec.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libbrotlidec.so.1
libbrotlicommon.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libbrotlicommon.so.1
liborc.so => /home/antoine/mambaforge/envs/pyarrow/lib/liborc.so
libprotobuf.so.32 => /home/antoine/mambaforge/envs/pyarrow/lib/libprotobuf.so.32
libutf8proc.so.2 => /home/antoine/mambaforge/envs/pyarrow/lib/libutf8proc.so.2
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
libgoogle_cloud_cpp_storage.so.2 => /home/antoine/mambaforge/envs/pyarrow/lib/libgoogle_cloud_cpp_storage.so.2
libbz2.so.1.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libbz2.so.1.0
liblz4.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/liblz4.so.1
libsnappy.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libsnappy.so.1
libz.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libz.so.1
libzstd.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libzstd.so.1
libaws-cpp-sdk-config.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-config.so
libaws-cpp-sdk-transfer.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-transfer.so
libaws-cpp-sdk-identity-management.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-identity-management.so
libaws-cpp-sdk-cognito-identity.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-cognito-identity.so
libaws-cpp-sdk-sts.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-sts.so
libaws-cpp-sdk-s3.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-s3.so
libaws-cpp-sdk-core.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-core.so
libre2.so.10 => /home/antoine/mambaforge/envs/pyarrow/lib/libre2.so.10
libabsl_str_format_internal.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_str_format_internal.so.2206.0.0
libgoogle_cloud_cpp_rest_internal.so.2 => /home/antoine/mambaforge/envs/pyarrow/lib/libgoogle_cloud_cpp_rest_internal.so.2
libgoogle_cloud_cpp_common.so.2 => /home/antoine/mambaforge/envs/pyarrow/lib/libgoogle_cloud_cpp_common.so.2
libabsl_time.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_time.so.2206.0.0
libabsl_strings.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_strings.so.2206.0.0
libabsl_strings_internal.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_strings_internal.so.2206.0.0
libabsl_int128.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_int128.so.2206.0.0
libabsl_civil_time.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_civil_time.so.2206.0.0
libabsl_time_zone.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_time_zone.so.2206.0.0
libabsl_bad_variant_access.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_bad_variant_access.so.2206.0.0
libabsl_base.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_base.so.2206.0.0
libabsl_spinlock_wait.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_spinlock_wait.so.2206.0.0
libabsl_bad_optional_access.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_bad_optional_access.so.2206.0.0
libabsl_throw_delegate.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_throw_delegate.so.2206.0.0
libabsl_raw_logging_internal.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_raw_logging_internal.so.2206.0.0
libabsl_log_severity.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_log_severity.so.2206.0.0
libcrc32c.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libcrc32c.so.1
libcurl.so.4 => /home/antoine/mambaforge/envs/pyarrow/lib/libcurl.so.4
libnghttp2.so.14 => /home/antoine/mambaforge/envs/pyarrow/lib/./libnghttp2.so.14
libssh2.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/./libssh2.so.1
libgssapi_krb5.so.2 => /home/antoine/mambaforge/envs/pyarrow/lib/./libgssapi_krb5.so.2
libkrb5.so.3 => /home/antoine/mambaforge/envs/pyarrow/lib/././libkrb5.so.3
libk5crypto.so.3 => /home/antoine/mambaforge/envs/pyarrow/lib/././libk5crypto.so.3
libcom_err.so.3 => /home/antoine/mambaforge/envs/pyarrow/lib/././libcom_err.so.3
libkrb5support.so.0 => /home/antoine/mambaforge/envs/pyarrow/lib/././libkrb5support.so.0
libkeyutils.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/././libkeyutils.so.1
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2
libaws-crt-cpp.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-crt-cpp.so
libaws-c-mqtt.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-mqtt.so.1.0.0
libaws-c-event-stream.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-event-stream.so.1.0.0
libaws-c-s3.so.0unstable => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-s3.so.0unstable
libaws-c-auth.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-auth.so.1.0.0
libaws-c-http.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-http.so.1.0.0
libaws-c-io.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-io.so.1.0.0
libs2n.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libs2n.so.1
libaws-c-compression.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-compression.so.1.0.0
libaws-c-cal.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-cal.so.1.0.0
libaws-c-sdkutils.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-sdkutils.so.1.0.0
libaws-checksums.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-checksums.so.1.0.0
libaws-c-common.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-common.so.1
libstdc++.so.6 => /home/antoine/mambaforge/envs/pyarrow/lib/libstdc++.so.6
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
libgcc_s.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libgcc_s.so.1
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
ld-linux-x86-64.so.2 => /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the failure with the reported CMake command in the Conda environment and inspect the generated link command for utils_test. Compare the CMake build-time library search paths with the paths shown by lddtree for libarrow.so.1300.0.0. Done means the test executables link successfully and the test build completes.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cmake
Domain
build-system, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.