cilium / cilium/proxy

Unable to compile on RedHat 9

Open
#1,247 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help wanted wait-for-upstream
Dominant language
C++
Stars
190
Forks
99
Avg merge
1d 5h
Merged PRs (30d)
30

Description

I'm trying to compile Cilium Proxy on RedHat UBI 9 and aarch64 architecture. To do so, I patched https://github.com/cilium/proxy/blob/main/bazel/toolchains/BUILD as shown below to adapt include directories and link options:

diff --git a/bazel/toolchains/BUILD b/bazel/toolchains/BUILD
index f0cbb5c3..be6322e7 100644
--- a/bazel/toolchains/BUILD
+++ b/bazel/toolchains/BUILD
@@ -49,10 +49,13 @@ cc_toolchain_config(
     coverage_link_flags = ["--coverage"],
     cpu = "aarch64",
     cxx_builtin_include_directories = [
-        "/usr/lib/llvm-17",
-        "/usr/aarch64-linux-gnu/include",
-        "/usr/include",
-    ],
+        "/usr/lib/clang/18/include",
+        "/usr/include",
+        "/opt/rh/gcc-toolset-13/root/usr/local/include",
+        "/opt/rh/gcc-toolset-13/root/usr/lib/gcc/aarch64-redhat-linux/13/include",
+        "/opt/rh/gcc-toolset-13/root/usr/include/c++/13",
+        "/opt/rh/gcc-toolset-13/root/usr/include/c++/13/aarch64-redhat-linux",
+    ],
     cxx_flags = ["-std=c++0x"],
     dbg_compile_flags = ["-g"],
     host_system_name = "local",
@@ -61,6 +64,8 @@ cc_toolchain_config(
         "-fuse-ld=lld",
         "-Wl,-no-as-needed",
         "-Wl,-z,relro,-z,now",
+        "-L/opt/rh/gcc-toolset-13/root/usr/lib/gcc/aarch64-redhat-linux/13",
+        "-Wl,-rpath,/opt/rh/gcc-toolset-13/root/usr/lib/gcc/aarch64-redhat-linux/13",
         "-lm",
     ],
     link_libs = ["-l:libstdc++.a"],
@@ -77,18 +82,18 @@ cc_toolchain_config(
     target_libc = "glibc",
     target_system_name = "aarch64-linux-gnu",
     tool_paths = {
-        "ar": "/usr/bin/llvm-ar-17",
-        "compat-ld": "/usr/bin/lld-17",
-        "ld": "/usr/bin/lld-17",
-        "gold": "/usr/bin/lld-17",
-        "cpp": "/usr/bin/clang-cpp-17",
-        "gcc": "/usr/bin/clang-17",
-        "dwp": "/usr/bin/llvm-dwp-17",
-        "gcov": "/usr/bin/llvmcov-17",
-        "nm": "/usr/bin/llvm-nm-17",
-        "objcopy": "/usr/bin/llvm-objcopy-17",
-        "objdump": "/usr/bin/llvm-objdump-17",
-        "strip": "/usr/bin/llvm-strip-17",
+        "ar": "/usr/bin/llvm-ar",
+        "compat-ld": "/usr/bin/lld",
+        "ld": "/usr/bin/lld",
+        "gold": "/usr/bin/lld",
+        "cpp": "/usr/bin/clang-cpp",
+        "gcc": "/usr/bin/clang",
+        "dwp": "/usr/bin/llvm-dwp",
+        "gcov": "/usr/bin/llvm-cov",
+        "nm": "/usr/bin/llvm-nm",
+        "objcopy": "/usr/bin/llvm-objcopy",
+        "objdump": "/usr/bin/llvm-objdump",
+        "strip": "/usr/bin/llvm-strip",
     },
     toolchain_identifier = "linux_aarch64",
     unfiltered_compile_flags = [

I have RedHat UBI 9 environment where I installed some system packages & bazellisk as shown below

dnf update -y
dnf --allowerasing install -y java-1.8.0-openjdk-headless clang-devel llvm-devel lld libstdc++-static

git clone git@github.com:cilium/proxy.git && cd proxy
./tools/install_bazelisk.sh

Commands I run:

./bazel/setup_clang.sh /usr
echo "build --config=libc++" > .bazelrc
bazel build \
      --config=release \
      --strip=always \
      --verbose_failures \
      --extra_toolchains=//bazel/toolchains:all \
      --noincompatible_strict_action_env \
      //:cilium-envoy

The error I'm facing:

ERROR: /home/cilium/.cache/bazel/_bazel_cilium/0211e60aab28805245749726eda1a88a/external/com_google_cel_cpp/common/BUILD:76:11: Compiling common/json.cc failed: (Exit 1): clang failed: error executing command (from target @com_google_cel_cpp//common:json) 
   (cd /home/cilium/.cache/bazel/_bazel_cilium/0211e60aab28805245749726eda1a88a/sandbox/processwrapper-sandbox/1372/execroot/cilium && \
   exec env - \
     BAZEL_COMPILER=clang \
     BAZEL_CXXOPTS='-stdlib=libc++' \
     BAZEL_LINKLIBS=-l%:libc++.a:-l%:libc++abi.a \
     BAZEL_LINKOPTS=-lm:-pthread \
     CC=clang \
     CXX=clang++ \
     CXXFLAGS='-stdlib=libc++' \
     LDFLAGS='-stdlib=libc++' \
     LLVM_CONFIG=/usr/bin/llvm-config \
    PATH=/usr/bin:/root/.local/bin:/root/bin:/usr/share/Modules/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    PWD=/proc/self/cwd \
  /usr/bin/clang '--target=aarch64-unknown-linux-gnu' '-fuse-ld=lld' -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wthread-safety-analysis -Wno-free-nonheap-object -fno-omit-frame-pointer -Wno-unused-command-line-argument -Wno-deprecated-builtins -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/aarch64-opt/bin/external/com_google_cel_cpp/common/_objs/json/json.d '-frandom-seed=bazel-out/aarch64-opt/bin/external/com_google_cel_cpp/common/_objs/json/json.o' '-DBAZEL_CURRENT_REPOSITORY="com_google_cel_cpp"' -iquote external/com_google_cel_cpp -iquote bazel-out/aarch64-opt/bin/external/com_google_cel_cpp -iquote external/com_google_absl -iquote bazel-out/aarch64-opt/bin/external/com_google_absl '-DABSL_MIN_LOG_LEVEL=4' -fdebug-types-section -fPIC -Wno-deprecated-declarations '-std=c++20' -fsized-deallocation -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c external/com_google_cel_cpp/common/json.cc -o bazel-out/aarch64-opt/bin/external/com_google_cel_cpp/common/_objs/json/json.o)
 # Configuration: 2f125838129c3a083292d6497d46584ac8a3a65159ebf628b0573bf0d7d63ead
 # Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
 In file included from external/com_google_cel_cpp/common/json.cc:15:
 In file included from external/com_google_cel_cpp/common/json.h:20:
 In file included from /opt/rh/gcc-toolset-13/root/usr/lib/gcc/aarch64-redhat-linux/13/../../../../include/c++/13/utility:69:
 In file included from /opt/rh/gcc-toolset-13/root/usr/lib/gcc/aarch64-redhat-linux/13/../../../../include/c++/13/bits/stl_pair.h:60:
 /opt/rh/gcc-toolset-13/root/usr/lib/gcc/aarch64-redhat-linux/13/../../../../include/c++/13/type_traits:1326:23: error: incomplete type 'cel::JsonObject' used in type trait expression
  1326 |                     __bool_constant<__has_trivial_destructor(_Tp)>>::type
       |                                     ^
 /opt/rh/gcc-toolset-13/root/usr/lib/gcc/aarch64-redhat-linux/13/../../../../include/c++/13/type_traits:3297:5: note: in instantiation of template class 'std::is_trivially_destructible<cel::JsonObject>' requested here
  3297 |     is_trivially_destructible<_Tp>::value;
       |     ^
 /opt/rh/gcc-toolset-13/root/usr/lib/gcc/aarch64-redhat-linux/13/../../../../include/c++/13/variant:367:5: note: in instantiation of variable template specialization 'std::is_trivially_destructible_v<cel::JsonObject>' requested here
   367 |           (is_trivially_destructible_v<_Types> && ...);
       |            ^
 /opt/rh/gcc-toolset-13/root/usr/lib/gcc/aarch64-redhat-linux/13/../../../../include/c++/13/variant:376:4: note: in instantiation of static data member 'std::__detail::__variant::_Traits<cel::JsonNull, bool, double, absl::Cord, cel::JsonArray, cel::JsonObject>::_S_trivial_dtor' requested here
   376 |           _S_trivial_dtor && _S_trivial_move_ctor
       |

(... MORE AND MORE SIMILAR ERRORS ...)

fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
Target //:cilium-envoy failed to build

Note: similar errors faced when compiling on x86_64 architecture.

Do you know how can I solve this?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure with the provided Bazel command and inspect bazel/toolchains/BUILD together with bazel/setup_clang.sh. Compare the configured toolchain and the RedHat UBI 9 compiler environment while tracing the reported external/com_google_cel_cpp/common/json.cc failure. Done means //:cilium-envoy builds successfully on the affected architectures.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
build-system, compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.