apple / apple/foundationdb

Recently building from source on MacOS arm64

Open
#12,380 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

Hi, I'm completely new to this project.
I am trying to build this project because I am team matching with a related job.
Not sure if this issues section is good for a write up, but I did run into issues following the docs.

My computer:
```
$ uname -a
Darwin Nucleus.local 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:19:22 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T8112 arm64
```

## TL;DR

For those who are also new, I think the docker image works the best, commands I ran were

```
docker pull foundationdb/build:rockylinux9_arm64-latest
docker run -it foundationdb/build:rockylinux9-latest /bin/bash

# Inside docker image
source /opt/rh/gcc-toolset-13/enable

mkdir -p src/foundationdb
git clone https://github.com/apple/foundationdb.git src/foundationdb/
mkdir build_output

cmake -S src/foundationdb -B build_output -G Ninja
ninja -C -j1
```

### Docker Issues
#### Running out of memory.

After running ninja, I got some failures like this:
```
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
```

It doesn't directly say running out of memory, but in README.md there is a hint: `Building FoundationDB requires at least 8GB of memory. More memory is needed when building in parallel. If the computer freezes or crashes, consider disabling parallelized build using ninja -j1.`

So I tried, and passed the build.

#### Using the wrong image
There are 2 images on foundationdb/build

foundationdb/build:rockylinux9-latest
foundationdb/build:rockylinux9_arm64-latest

I didn't notice using the wrong one at first, and got the following errors.

```
ninja: Entering directory `build_output'
[0/2] Re-checking globbed directories...
[1/1825] Compile actor compiler
FAILED: actorcompiler.exe /tmp/src/foundationdb/build_output/actorcompiler.exe
cd /tmp/src/foundationdb/build_output && /usr/bin/mcs -r:System,System.Core,System.Xml.Linq,System.Data.DataSetExtensions,Microsoft.CSharp,System.Data,System.Xml /tmp/src/foundationdb/src/foundationdb/flow/actorcompiler/ActorCompiler.cs /tmp/src/foundationdb/src/foundationdb/flow/actorcompiler/ActorParser.cs /tmp/src/foundationdb/src/foundationdb/flow/actorcompiler/ParseTree.cs /tmp/src/foundationdb/src/foundationdb/flow/actorcompiler/Program.cs /tmp/src/foundationdb/src/foundationdb/flow/actorcompiler/Properties/AssemblyInfo.cs -target:exe -out:actorcompiler.exe
* Assertion: should not be reached at tramp-amd64.c:607

=================================================================
Native Crash Reporting
=================================================================
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================

=================================================================
Native stacktrace:
=================================================================
0x5555555ff75a - /usr/bin/mono :
0x5555555c8b3e - /usr/bin/mono :
```

### Mac Issues

Mac builds, but there are stuff to fix. I haven't run the tests yet, because python is messed up for me.

#### toml11 warnings

toml11 has a warning that actually fails the ninja build.
```
loading initial cache file /Users/katieliu/code/fdb_build/build3/toml11Project-prefix/tmp/toml11Project-cache-Release.cmake
CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 3.5 has been removed from CMake.

Update the VERSION argument value. Or, use the ... syntax
to tell CMake that the project requires at least but has been updated
to work with policies introduced by or earlier.

Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.

-- Configuring incomplete, errors occurred!
```
Fix: Replace the first line in CMakeLists.
In the build directory, edit
`toml11Project-prefix/src/toml11Project/CMakeLists.txt`
```
- cmake_minimum_required(VERSION 3.1)
+ cmake_minimum_required(VERSION 3.5)
```

It seems to be only a warning, but after this fix, the build continues.
#### Installed Boost with brew
README.md says this is possible, but it didn't work for me.

When I ran cmake, I see that the local version isn't used.
```
-- Using Clang version of boost
-- Didn't find Boost -- will compile from source
-- Use clang to build boost
```

but later it interferes with the build
```
/Users/katieliu/code/fdb_build/foundationdb/flow/IThreadPool.cpp:70:15: error: no type named 'io_service' in namespace 'boost::asio'
boost::asio::io_service ios;
~~~~~~~~~~~~~^
/Users/katieliu/code/fdb_build/foundationdb/flow/IThreadPool.cpp:71:15: error: no member named 'io_service' in namespace 'boost::asio'
boost::asio::io_service::work dontstop;
~~~~~~~~~~~~~^
/Users/katieliu/code/fdb_build/foundationdb/flow/IThreadPool.cpp:101:33: error: cannot initialize object parameter of type 'const ThreadUnsafeReferenceCounted' with an expression of type 'ThreadPool'
ReferenceCounted::addref();
^~~~~~
/Users/katieliu/code/fdb_build/foundationdb/flow/IThreadPool.cpp:108:33: error: cannot initialize object parameter of type 'const ThreadUnsafeReferenceCounted' with an expression of type 'ThreadPool'
ReferenceCounted::delref();
^~~~~~
/Users/katieliu/code/fdb_build/foundationdb/flow/IThreadPool.cpp:113:57: error: cannot initialize object parameter of type 'const ThreadUnsafeReferenceCounted' with an expression of type 'ThreadPool'
void addref() override { ReferenceCounted::addref(); }
^~~~~~
/Users/katieliu/code/fdb_build/foundationdb/flow/IThreadPool.cpp:115:37: error: cannot initialize object parameter of type 'const ThreadUnsafeReferenceCounted' with an expression of type 'ThreadPool'
if (ReferenceCounted::delref_no_destroy()) {
^~~~~~~~~~~~~~~~~
/Users/katieliu/code/fdb_build/foundationdb/flow/IThreadPool.cpp:129:9: error: no matching conversion for functional-style cast from 'ThreadPool *' to 'Reference'
return Reference(new ThreadPool(stackSize, pri));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/katieliu/code/fdb_build/foundationdb/flow/include/flow/FastRef.h:104:11: note: candidate constructor not viable: no known conversion from 'ThreadPool *' to 'IThreadPool *' for 1st argument
explicit Reference(P* ptr) : ptr(ptr) {}
^
/Users/katieliu/code/fdb_build/foundationdb/flow/include/flow/FastRef.h:110:2: note: candidate constructor not viable: no known conversion from 'ThreadPool *' to 'const Reference' for 1st argument
Reference(const Reference& r) : ptr(r.getPtr()) {
^
/Users/katieliu/code/fdb_build/foundationdb/flow/include/flow/FastRef.h:114:2: note: candidate constructor not viable: no known conversion from 'ThreadPool *' to 'Reference' for 1st argument
Reference(Reference&& r) noexcept : ptr(r.getPtr()) { r.ptr = nullptr; }
^
/Users/katieliu/code/fdb_build/foundationdb/flow/include/flow/FastRef.h:117:2: note: candidate template ignored: could not match 'Reference' against 'ThreadPool *'
Reference(const Reference& r) : ptr(r.getPtr()) {
^
/Users/katieliu/code/fdb_build/foundationdb/flow/include/flow/FastRef.h:122:2: note: candidate template ignored: could not match 'Reference' against 'ThreadPool *'
Reference(Reference&& r) : ptr(r.getPtr()) {
^
/Users/katieliu/code/fdb_build/foundationdb/flow/include/flow/FastRef.h:103:2: note: candidate constructor not viable: requires 0 arguments, but 1 was provided
Reference() : ptr(nullptr) {}
^
7 errors generated.
ninja: build stopped: subcommand failed.
```

This seems to be a boost version problem

```
ChatGPT:
That error usually comes up because recent versions of Boost.Asio removed boost::asio::io_service in favor of boost::asio::io_context.

In older Boost (≤ 1.66), the main I/O execution context was called boost::asio::io_service.
Since Boost 1.70, io_service was officially replaced by boost::asio::io_context. They are nearly identical in functionality, but io_context is the preferred name now.
```

Since brew only has new versions for me now, I give up here

```
brew search boost
==> Formulae
boost@1.76 boost@1.85
```

Uninstalling brew makes the build succeed

```
brew uninstall boost
```

#### ctest

I could finish the build, and also successfully package.
```
/packaging/osx/buildpkg.sh .
```
Problem is most tests fail using
```
ctest
```

```
4/59 Test #4: test_venv_setup .....................................................***Failed 17.93 sec
Start 5: single_process_fdbcli_tests
Failed test dependencies: test_venv_setup
5/59 Test #5: single_process_fdbcli_tests .........................................***Not Run 0.00 sec
```

I'm sure it's because python is messed up on my computer. But docker was succeeding so I switched over.

Contributor guide

Open the contributing guide

Research direction

Start with README.md and reproduce the reported macOS arm64 build using the documented Docker images and CMake/Ninja commands. Then inspect the toml11 configuration and the ctest failure for test_venv_setup. Done means the documented build path and relevant tests work without the reported dependency or platform failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, docker, python
Domain
build-system, operating-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.