bazelbuild / bazelbuild/rules_rust
Linking with clang on aarch64-linux-android results in undefined symbols
- Dominant language
- Starlark
- Stars
- 843
- Forks
- 651
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 15
Description
We have a fairly large Bazel build which generates a .so for use with Android. Incorporating rules_rust into the mix (even as simple as adding a single rust_library with no addtional dependencies causes the resulting .so to contain two undefined symbols:
```
U _ZN6memchr6memchr8fallback6memchr17h69d58a5ccf33f3aaE
U _ZN6memchr6memchr8fallback7memchr217h3fdce6c95123ced9E
```
Looking at one of them, I tracked these down to being referenced and defined by the vendored rustlibs:
Referenced here:
```
➜ rustlib objdump -x aarch64-linux-android/lib/libobject-e4d8140a90db47ce.rlib|grep _ZN6memchr6memchr8fallback6memchr17h69d58a5ccf33f3aaE
0000000000000000 *UND* 0000000000000000 _ZN6memchr6memchr8fallback6memchr17h69d58a5ccf33f3aaE
000000000000003c R_AARCH64_CALL26 _ZN6memchr6memchr8fallback6memchr17h69d58a5ccf33f3aaE
000000000000001c R_AARCH64_CALL26 _ZN6memchr6memchr8fallback6memchr17h69d58a5ccf33f3aaE
```
Defined here:
```
➜ rustlib objdump -d aarch64-linux-android/lib/libmemchr-7409783eefd1e836.rlib|grep _ZN6memchr6memchr8fallback6memchr17h69d58a5ccf33f3aaE
Disassembly of section .text._ZN6memchr6memchr8fallback6memchr17h69d58a5ccf33f3aaE:
0000000000000000 <_ZN6memchr6memchr8fallback6memchr17h69d58a5ccf33f3aaE>:
4: 62 01 00 54 b.hs 0x30 <_ZN6memchr6memchr8fallback6memchr17h69d58a5ccf33f3aaE+0x30>
8: 02 09 00 b4 cbz x2, 0x128 <_ZN6memchr6memchr8fallback6memchr17h69d58a5ccf33f3aaE+0x128>
```
From what I can tell, the linker decides to include the reference (libobject) but does not include the definition (libmemchr).
Looking at the linker args nothing special is done wrt these libraries:
```
external/androidndk/ndk/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang
-shared
-o
-Wl,-whole-archive
bazel-out/android-arm64-v8a-fastbuild/bin/external/envoy_mobile/library/common/jni/libenvoy_jni_lib.lo
-Wl,-no-whole-archive
bazel-out/android-arm64-v8a-fastbuild/bin/external/envoy_mobile/library/common/jni/libndk_jni_support.a
-Wl,-whole-archive
bazel-out/android-arm64-v8a-fastbuild/bin/src/cc/libjni_interface.lo
-Wl,-no-whole-archive
bazel-out/android-arm64-v8a-fastbuild/bin/src/cc/libjni_interface_rs-131865085.a
-Wl,-whole-archive
bazel-out/android-arm64-v8a-fastbuild/bin/src/cc/librust_alloc.lo
-Wl,-no-whole-archive
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/librustc_std_workspace_alloc-e69adea976218243.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/librustc_std_workspace_core-ed1c6adb8e36cabe.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/librustc_std_workspace_std-4294fd5280257638.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libstd-af8cca4ce4f6804f.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libstd_detect-0e975b57636672c8.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libaddr2line-e5d6d51ed5bbf971.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libcfg_if-a5e658c5749822f1.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libgetopts-f5d93f2c3e5a88d2.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libgimli-70667b4716457f81.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libhashbrown-82f9c426c72474a5.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/liblibc-cf442dcec02ecfaf.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libmemchr-7409783eefd1e836.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libminiz_oxide-0e0429dadfd01176.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libobject-e4d8140a90db47ce.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libpanic_unwind-d20b7629bade7b19.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libproc_macro-fb08dac290c4c7f1.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libprofiler_builtins-09579e987bfb7182.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/librustc_demangle-e513a68fe4301435.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libtest-97ee2a167210714e.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libunicode_width-4d4139559e030181.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libunwind-5d21e2a63fe6a0f1.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libadler-f1884c9b41122156.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libcore-669955cdd70efb76.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/libcompiler_builtins-86e41092bca9bdf8.a
bazel-out/android-arm64-v8a-fastbuild/bin/external/rust_darwin_x86_64/lib/rustlib/aarch64-linux-android/lib/liballoc-0a2707a8771b9442.a
```
I was able to get things working by force linking the stdlib libraries (with the exclusion of the profiler, which resulted in issues resolving the `__start___llvm_prf_vnds` during dlopen):
```
diff --git a/rust/toolchain.bzl b/rust/toolchain.bzl
index 687e0df..d4cc7fc 100644
--- a/rust/toolchain.bzl
+++ b/rust/toolchain.bzl
@@ -92,6 +92,7 @@ def _ltl(library, ctx, cc_toolchain, feature_configuration):
actions = ctx.actions,
feature_configuration = feature_configuration,
cc_toolchain = cc_toolchain,
+ alwayslink = ("profiler" not in library.basename),
static_library = library,
pic_static_library = library,
)
```
So with all this: I'm not sure what the underlying reason behind the linker omitting libmemchr is, nor whether force linking the rustlibs when compiling with a CC toolchain is the right thing to do here. Digging into memchr it seems like perhaps we shouldn't even be using the fallback calls when libc is available, so perhaps this is an issue with the vendored rustlib on aarch64-linux-android.
For reference I have not verified whether this works on other Android platforms, and this is happening rules_rust 0.2.1
Contributor guide
Assessment
This issue has not been assessed yet.