apple / apple/foundationdb

Unable to build RocksDB on main; finds non-existent library

Open
#11,826 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

I'm building from main `e19df49a` on an M1 macOS 15.1.1 with llvm/clang-19. (I have the same problem on 7.3.6.) I don't have any CMAKE environment variables set.

I receive the following error:

`The source directory ...fdbserver/rocksdb-prefix/src/rocksdb" does not appear to contain CMakeLists.txt`

```
[219/1746] Performing configure step for 'rocksdb'
FAILED: fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure
cd /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-build && /opt/homebrew/bin/cmake
-DUSE_RTTI=1
-DPORTABLE=1
-DFORCE_SSE42=
-DFORCE_AVX=
-DFORCE_AVX2=
-DCMAKE_C_COMPILER=/opt/homebrew/opt/llvm/bin/clang
-DCMAKE_C_FLAGS=
-DCMAKE_CXX_STANDARD=20
-DCMAKE_CXX_COMPILER=/opt/homebrew/opt/llvm/bin/clang++
"-DCMAKE_CXX_FLAGS= -stdlib=libc++"
-DCMAKE_SHARED_LINKER_FLAGS=
-DCMAKE_STATIC_LINKER_FLAGS=
-DCMAKE_EXE_LINKER_FLAGS=
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_EXPORT_COMPILE_COMMANDS=
-DFAIL_ON_WARNINGS=OFF
-DWITH_GFLAGS=OFF
-DWITH_TESTS=OFF
-DWITH_TOOLS=OFF
-DWITH_CORE_TOOLS=OFF
-DWITH_BENCHMARK_TOOLS=OFF
-DWITH_BZ2=OFF
-DWITH_LZ4=ON
-DWITH_SNAPPY=OFF
-DWITH_ZLIB=OFF
-DWITH_ZSTD=OFF
-DWITH_LIBURING=OFF
-DWITH_TSAN=OFF
-DWITH_ASAN=OFF
-DWITH_UBSAN=OFF
-DROCKSDB_BUILD_SHARED=OFF
-DCMAKE_POSITION_INDEPENDENT_CODE=True
-GNinja
-S /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb
-B /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-build && /opt/homebrew/bin/cmake
-E touch /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure
```

```
CMake Error: The source directory "/Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb" does not appear to contain CMakeLists.txt.
```

Cmake finds rocksdb; however, it says it's finding the library in an empty directory in the build tree and not the local `/opt/homebrew/lib/librocksdb.a` location.

```
-- Found RocksDB version: 9.8.4
-- Found RocksDB: /opt/homebrew/include (found suitable version "9.8.4", minimum required is "9.7.3")
-- Found RocksDB library: /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-build/librocksdb.a
-- Found RocksDB includes: /opt/homebrew/include
```
This directory--`/Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-build`--is empty.

```
[jayed@merlot] ls -al /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-build/

drwxr-xr-x - jayed 11 Dec 14:52 ./
drwxr-xr-x - jayed 11 Dec 14:52 ../
```
The only cmake references to `librocksdb.a` are here:
```
cmake/CompileRocksDB.cmake
47: BUILD_BYPRODUCTS /librocksdb.a
53: ${BINARY_DIR}/librocksdb.a)
59: BUILD_BYPRODUCTS /librocksdb.a
65: ${BINARY_DIR}/librocksdb.a)
```

`cmake -G Ninja -DBUILD_JAVA_BINDING=off -DCMAKE_OSX_SYSROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk ~/repos/foundationdb`

```
-- =========================================
-- Components Build Overview
-- =========================================
-- Build Bindings (depends on Python): ON
-- Build C Bindings: ON
-- Build Python Bindings: ON
-- Build Java Bindings: OFF
-- Build Go bindings: ON
-- Build Ruby bindings: ON
-- Build Swift (depends on Swift): OFF
-- Build Documentation (make html): ON
-- Build Python sdist (make package): ON
-- Configure CTest (depends on Python): ON
-- Build with RocksDB: ON
-- Build with AWS SDK: OFF
-- =========================================
```

Here's the full rocksdb directory tree
```
fdbserver/rocksdb-prefix
├── src
│ ├── rocksdb
│ ├── rocksdb-build
│ └── rocksdb-stamp
│ ├── rocksdb-custominfo.txt
│ ├── rocksdb-download
│ ├── rocksdb-mkdir
│ ├── rocksdb-patch
│ ├── rocksdb-patch-info.txt
│ ├── rocksdb-update
│ └── rocksdb-update-info.txt
└── tmp
├── rocksdb-cfgcmd.txt
└── rocksdb-mkdirs.cmake
```
Here are the local rocksdb libraries
```
[jayed@merlot ~repos/foundationdb](main) eza -l /opt/homebrew/lib/libroc*

/opt/homebrew/lib/librocksdb.9.8.4.dylib -> ../Cellar/rocksdb/9.8.4/lib/librocksdb.9.8.4.dylib

/opt/homebrew/lib/librocksdb.9.dylib -> ../Cellar/rocksdb/9.8.4/lib/librocksdb.9.dylib

/opt/homebrew/lib/librocksdb.a -> ../Cellar/rocksdb/9.8.4/lib/librocksdb.a

/opt/homebrew/lib/librocksdb.dylib -> ../Cellar/rocksdb/9.8.4/lib/librocksdb.dylib
```

The cmake command includes `-E touch /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure`. That file does not exist; the parent directory does. The directory has to correct permissions.

```
[jayed@merlot] ls Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure

"Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure": No such file or directory (os error 2)
```

Contributor guide

Open the contributing guide

Research direction

Start with cmake/CompileRocksDB.cmake and the generated RocksDB configure command shown in the report. Reproduce the macOS/Ninja configure step and inspect the rocksdb-prefix/src/rocksdb and rocksdb-build directories. Done means the dependency source is configured from a valid checkout and the FoundationDB build no longer points at an empty RocksDB directory.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.