bazel-contrib / bazel-contrib/rules_foreign_cc
How to use the cmake rules and local nvcc compiler to build CUDA project?
- Dominant language
- Starlark
- Stars
- 737
- Forks
- 270
- PR merge metrics
- No merged PRs in 30d
Description
I have a cuda project like https://github.com/NVIDIA-Merlin/HierarchicalKV/ , I can build it with it's cmake build guide.
Now I want to use this repo in my Bazel project, so I choose rules_foreign_cc's cmake rule to build it.
Here is my scripts,
WORKSPACE file
```
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "rules_foreign_cc",
sha256 = "a2e6fb56e649c1ee79703e99aa0c9d13c6cc53c8d7a0cbb8797ab2888bbc99a3",
strip_prefix = "rules_foreign_cc-0.12.0",
url = "https://github.com/bazelbuild/rules_foreign_cc/releases/download/0.12.0/rules_foreign_cc-0.12.0.tar.gz",
)
load("@rules_foreign_cc//foreign_cc:repositories.bzl", "rules_foreign_cc_dependencies")
# This sets up some common toolchains for building targets. For more details, please see
# https://bazelbuild.github.io/rules_foreign_cc/0.12.0/flatten.html#rules_foreign_cc_dependencies
rules_foreign_cc_dependencies()
load("@bazel_features//:deps.bzl", "bazel_features_deps")
bazel_features_deps()
http_archive(
name = "HierarchicalKV",
build_file = "hkv.BUILD",
sha256 = "a73d7bea159173db2038f7c5215a7d1fbd5362adfb232fabde206dc64a1e817c",
strip_prefix = "HierarchicalKV-0.1.0-beta.12",
url = "https://github.com/NVIDIA-Merlin/HierarchicalKV/archive/refs/tags/v0.1.0-beta.12.tar.gz",
)
```
hkv.BUILD
```
load("@rules_foreign_cc//foreign_cc:defs.bzl", "cmake")
package(default_visibility = ["//visibility:public"])
licenses(["notice"]) # BSD 3-Clause
exports_files(["LICENSE"])
filegroup(
name = "all_srcs",
srcs = glob(["**"]),
)
cmake(
name = "hkv",
build_args = [
"--verbose",
"-j `nproc`",
],
cache_entries = {
"CMAKE_BUILD_TYPE": "Release",
"BUILD_TESTS": "OFF",
"BUILD_BENCHMARKS": "OFF",
"sm": "80",
},
lib_source = ":all_srcs",
)
```
BUILD file:
```
load("@bazel_skylib//rules:build_test.bzl", "build_test")
build_test(
name = "build_test",
targets = [
"@HierarchicalKV//:hkv",
],
)
```
When I build the target with `bazel build build_test`, I got this error:
```
INFO: Found 1 target...
ERROR: /workspaces/cache/bazel/external/HierarchicalKV/BUILD.bazel:14:6: Foreign Cc - CMake: Building hkv failed: (Exit 1): process-wrapper failed: error executing command
(cd /workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray && \
exec env - \
CLANG_CUDA_COMPILER_PATH=/usr/lib/llvm-17/bin/clang \
HTTPS_PROXY=http://127.0.0.1:7890 \
HTTP_PROXY=http://127.0.0.1:7890 \
LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/:/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 \
PATH=/root/.cache/bazelisk/downloads/sha256/a40ac69263440761199fcb8da47ad4e3f328cbe79ffbf4ecc14e5ba252857307/bin:/bin:/opt/conda/bin:/opt/conda/condabin:/opt/conda/bin:/usr/l
ocal/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
PYTHON_BIN_PATH=/bin/python3 \
PYTHON_LIB_PATH=/usr/lib/python3/dist-packages \
TF2_BEHAVIOR=1 \
TMPDIR=/tmp \
/root/.cache/bazel/_bazel_root/install/20da5ab742b8d3d499c34fdafcd3c8b8/process-wrapper '--timeout=0' '--kill_delay=15' '--stats=/workspaces/cache/bazel/sandbox/processwrapper-
sandbox/19/stats.out' /bin/bash -c bazel-out/k8-opt/bin/external/HierarchicalKV/hkv_foreign_cc/wrapper_build_script.sh)
rules_foreign_cc: Build failed!
rules_foreign_cc: Keeping temp build directory and dependencies directory for debug.
rules_foreign_cc: Please note that the directories inside a sandbox are still cleaned unless you specify --sandbox_debug Bazel command line flag.
rules_foreign_cc: Printing build logs:
_____ BEGIN BUILD LOGS _____
Bazel external C/C++ Rules. Building library hkv
Environment:______________
BUILD_SCRIPT=bazel-out/k8-opt/bin/external/HierarchicalKV/hkv_foreign_cc/build_script.sh
EXT_BUILD_ROOT=/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray
BUILD_LOG=bazel-out/k8-opt/bin/external/HierarchicalKV/hkv_foreign_cc/CMake.log
PWD=/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray
BUILD_WRAPPER_SCRIPT=bazel-out/k8-opt/bin/external/HierarchicalKV/hkv_foreign_cc/wrapper_build_script.sh
PYTHON_BIN_PATH=/bin/python3
TMPDIR=/tmp
EXT_BUILD_DEPS=/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt/bin/external/HierarchicalKV/hkv.ext_build_deps
BUILD_TMPDIR=/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt/bin/external/HierarchicalKV/hkv.build_tmpdir
SHLVL=2
HTTPS_PROXY=http://127.0.0.1:7890
HTTP_PROXY=http://127.0.0.1:7890
LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/:/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
CLANG_CUDA_COMPILER_PATH=/usr/lib/llvm-17/bin/clang
INSTALLDIR=/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt/bin/external/HierarchicalKV/hkv
PYTHON_LIB_PATH=/usr/lib/python3/dist-packages
TF2_BEHAVIOR=1
PATH=/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray:/root/.cache/bazelisk/downloads/sha256/a40ac69263440761199fcb8da47ad4e3f328cbe79ffbf4ecc14e5ba2528
57307/bin:/bin:/opt/conda/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/bin/env
__________________________
+ /workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/external/cmake-3.22.1-linux-x86_64/bin/cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTS=OFF -D[21/1930]
CHMARKS=OFF -Dsm=80 -DCMAKE_TOOLCHAIN_FILE=crosstool_bazel.cmake -DCMAKE_INSTALL_PREFIX=/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-op
t/bin/external/HierarchicalKV/hkv -DCMAKE_PREFIX_PATH=/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt/bin/external/HierarchicalKV/hkv.
ext_build_deps -DCMAKE_RANLIB= -DCMAKE_MAKE_PROGRAM=/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt-exec-50AE0418/bin/external/rules_f
oreign_cc/toolchains/make/bin/make -G 'Unix Makefiles' /workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/external/HierarchicalKV
-- The CXX compiler identification is Clang 17.0.6
-- The CUDA compiler identification is NVIDIA 12.2.140
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/lib/llvm-17/bin/clang - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - failed
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - broken
CMake Error at /workspaces/cache/bazel/external/cmake-3.22.1-linux-x86_64/share/cmake-3.22/Modules/CMakeTestCUDACompiler.cmake:56 (message):
The CUDA compiler
"/usr/local/cuda/bin/nvcc"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt/bin/external/HierarchicalKV/hkv.build_tmpdir/CMakeFiles/CMakeTmp
Run Build Command(s):/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt-exec-50AE0418/bin/external/rules_foreign_cc/toolchains/make/b
in/make -f Makefile cmTC_1738c/fast && /workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt-exec-50AE0418/bin/external/rules_foreign_cc/too
lchains/make/bin/make -f CMakeFiles/cmTC_1738c.dir/build.make CMakeFiles/cmTC_1738c.dir/build
make[1]: Entering directory '/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt/bin/external/HierarchicalKV/hkv.build_tmpdir/CMakeFil
es/CMakeTmp'
Building CUDA object CMakeFiles/cmTC_1738c.dir/main.cu.o
/usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -MD -MT CMakeFiles/cmTC_1738c.dir/main.cu.o -MF CMakeFiles/cmTC_1738c.dir/main.cu.o.d -x cu -c /workspaces/cache
/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt/bin/external/HierarchicalKV/hkv.build_tmpdir/CMakeFiles/CMakeTmp/main.cu -o CMakeFiles/cmTC_1738c.dir/m
ain.cu.o
Linking CUDA executable cmTC_1738c
/workspaces/cache/bazel/external/cmake-3.22.1-linux-x86_64/bin/cmake -E cmake_link_script CMakeFiles/cmTC_1738c.dir/link.txt --verbose=1
/bin/g++ -Wl,-no-as-needed -lstdc++ -no-canonical-prefixes -pie -Wl,-z,relro,-z,now -Wl,--gc-sections -Wl,--build-id=md5 -Wl,--hash-style=gnu --cuda-path=/workspaces/cache/ba
zel/sandbox/processwrapper-sandbox/19/execroot/deepray/external/cuda_nvcc -fuse-ld=lld -lm -fuse-ld=lld -lm CMakeFiles/cmTC_1738c.dir/main.cu.o -o cmTC_1738c -lcudadevrt -lcuda
rt_static -lrt -lpthread -ldl -L"/usr/local/cuda/targets/x86_64-linux/lib/stubs" -L"/usr/local/cuda/targets/x86_64-linux/lib"
g++: error: unrecognized command-line option '--cuda-path=/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/external/cuda_nvcc'
make[1]: *** [CMakeFiles/cmTC_1738c.dir/build.make:100: cmTC_1738c] Error 1
make[1]: Leaving directory '/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt/bin/external/HierarchicalKV/hkv.build_tmpdir/CMakeFile
s/CMakeTmp'
make: *** [Makefile:127: cmTC_1738c/fast] Error 2
CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:17 (project)
-- Configuring incomplete, errors occurred!
See also "/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt/bin/external/HierarchicalKV/hkv.build_tmpdir/CMakeFiles/CMakeOutput.log".
See also "/workspaces/cache/bazel/sandbox/processwrapper-sandbox/19/execroot/deepray/bazel-out/k8-opt/bin/external/HierarchicalKV/hkv.build_tmpdir/CMakeFiles/CMakeError.log".
_____ END BUILD LOGS _____
rules_foreign_cc: Build wrapper script location: bazel-out/k8-opt/bin/external/HierarchicalKV/hkv_foreign_cc/wrapper_build_script.sh
rules_foreign_cc: Build script location: bazel-out/k8-opt/bin/external/HierarchicalKV/hkv_foreign_cc/build_script.sh
rules_foreign_cc: Build log location: bazel-out/k8-opt/bin/external/HierarchicalKV/hkv_foreign_cc/CMake.log
Target //:build_test failed to build
INFO: Elapsed time: 7.980s, Critical Path: 5.65s
INFO: 4 processes: 4 internal.
FAILED: Build did NOT complete successfully
```
So I want to know how to set my local nvcc compiler for the cmake rule
Other infomations:
OS version: Ubuntu 22.04
clang version: Ubuntu clang version 17.0.6
nvcc version: release 12.2, V12.2.140
bazel version: 6.5.0
Contributor guide
Research direction
Start with hkv.BUILD and the WORKSPACE configuration, then inspect CMakeLists.txt at line 17 and the generated CMake.log and CMakeError.log. Reproduce with the shown `bazel build build_test` command and determine how the CMake rule selects the local nvcc compiler; done means the CUDA compiler check and the external HierarchicalKV build succeed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake
- Domain
- build-system, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100