Unable to build RocksDB on main; finds non-existent library
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
I'm building from main `e19df49a` on an M1 macOS 15.1.1 with llvm/clang-19. (I have the same problem on 7.3.6.) I don't have any CMAKE environment variables set.
I receive the following error:
`The source directory ...fdbserver/rocksdb-prefix/src/rocksdb" does not appear to contain CMakeLists.txt`
```
[219/1746] Performing configure step for 'rocksdb'
FAILED: fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure
cd /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-build && /opt/homebrew/bin/cmake
-DUSE_RTTI=1
-DPORTABLE=1
-DFORCE_SSE42=
-DFORCE_AVX=
-DFORCE_AVX2=
-DCMAKE_C_COMPILER=/opt/homebrew/opt/llvm/bin/clang
-DCMAKE_C_FLAGS=
-DCMAKE_CXX_STANDARD=20
-DCMAKE_CXX_COMPILER=/opt/homebrew/opt/llvm/bin/clang++
"-DCMAKE_CXX_FLAGS= -stdlib=libc++"
-DCMAKE_SHARED_LINKER_FLAGS=
-DCMAKE_STATIC_LINKER_FLAGS=
-DCMAKE_EXE_LINKER_FLAGS=
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_EXPORT_COMPILE_COMMANDS=
-DFAIL_ON_WARNINGS=OFF
-DWITH_GFLAGS=OFF
-DWITH_TESTS=OFF
-DWITH_TOOLS=OFF
-DWITH_CORE_TOOLS=OFF
-DWITH_BENCHMARK_TOOLS=OFF
-DWITH_BZ2=OFF
-DWITH_LZ4=ON
-DWITH_SNAPPY=OFF
-DWITH_ZLIB=OFF
-DWITH_ZSTD=OFF
-DWITH_LIBURING=OFF
-DWITH_TSAN=OFF
-DWITH_ASAN=OFF
-DWITH_UBSAN=OFF
-DROCKSDB_BUILD_SHARED=OFF
-DCMAKE_POSITION_INDEPENDENT_CODE=True
-GNinja
-S /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb
-B /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-build && /opt/homebrew/bin/cmake
-E touch /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure
```
```
CMake Error: The source directory "/Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb" does not appear to contain CMakeLists.txt.
```
Cmake finds rocksdb; however, it says it's finding the library in an empty directory in the build tree and not the local `/opt/homebrew/lib/librocksdb.a` location.
```
-- Found RocksDB version: 9.8.4
-- Found RocksDB: /opt/homebrew/include (found suitable version "9.8.4", minimum required is "9.7.3")
-- Found RocksDB library: /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-build/librocksdb.a
-- Found RocksDB includes: /opt/homebrew/include
```
This directory--`/Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-build`--is empty.
```
[jayed@merlot] ls -al /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-build/
drwxr-xr-x - jayed 11 Dec 14:52 ./
drwxr-xr-x - jayed 11 Dec 14:52 ../
```
The only cmake references to `librocksdb.a` are here:
```
cmake/CompileRocksDB.cmake
47: BUILD_BYPRODUCTS /librocksdb.a
53: ${BINARY_DIR}/librocksdb.a)
59: BUILD_BYPRODUCTS /librocksdb.a
65: ${BINARY_DIR}/librocksdb.a)
```
`cmake -G Ninja -DBUILD_JAVA_BINDING=off -DCMAKE_OSX_SYSROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk ~/repos/foundationdb`
```
-- =========================================
-- Components Build Overview
-- =========================================
-- Build Bindings (depends on Python): ON
-- Build C Bindings: ON
-- Build Python Bindings: ON
-- Build Java Bindings: OFF
-- Build Go bindings: ON
-- Build Ruby bindings: ON
-- Build Swift (depends on Swift): OFF
-- Build Documentation (make html): ON
-- Build Python sdist (make package): ON
-- Configure CTest (depends on Python): ON
-- Build with RocksDB: ON
-- Build with AWS SDK: OFF
-- =========================================
```
Here's the full rocksdb directory tree
```
fdbserver/rocksdb-prefix
├── src
│ ├── rocksdb
│ ├── rocksdb-build
│ └── rocksdb-stamp
│ ├── rocksdb-custominfo.txt
│ ├── rocksdb-download
│ ├── rocksdb-mkdir
│ ├── rocksdb-patch
│ ├── rocksdb-patch-info.txt
│ ├── rocksdb-update
│ └── rocksdb-update-info.txt
└── tmp
├── rocksdb-cfgcmd.txt
└── rocksdb-mkdirs.cmake
```
Here are the local rocksdb libraries
```
[jayed@merlot ~repos/foundationdb](main) eza -l /opt/homebrew/lib/libroc*
/opt/homebrew/lib/librocksdb.9.8.4.dylib -> ../Cellar/rocksdb/9.8.4/lib/librocksdb.9.8.4.dylib
/opt/homebrew/lib/librocksdb.9.dylib -> ../Cellar/rocksdb/9.8.4/lib/librocksdb.9.dylib
/opt/homebrew/lib/librocksdb.a -> ../Cellar/rocksdb/9.8.4/lib/librocksdb.a
/opt/homebrew/lib/librocksdb.dylib -> ../Cellar/rocksdb/9.8.4/lib/librocksdb.dylib
```
The cmake command includes `-E touch /Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure`. That file does not exist; the parent directory does. The directory has to correct permissions.
```
[jayed@merlot] ls Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure
"Users/jayed/work/fdb-build/fdbserver/rocksdb-prefix/src/rocksdb-stamp/rocksdb-configure": No such file or directory (os error 2)
```
Contributor guide
Research direction
Start with cmake/CompileRocksDB.cmake and the generated RocksDB configure command shown in the report. Reproduce the macOS/Ninja configure step and inspect the rocksdb-prefix/src/rocksdb and rocksdb-build directories. Done means the dependency source is configured from a valid checkout and the FoundationDB build no longer points at an empty RocksDB directory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100