Linux arm64 16kb pages (jemalloc unsupported page size)
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
Hello, the binary builds for linux/arm64 don't run Apple's M1/M2/M3.
attempting to run `fdbmonitor` shows that fdbserver can't start because of the statically built jemalloc is compiled w/ 4kb pages.
Here's the error:
```
miguel@armadura fdb % doas ./fdbmonitor.aarch64
Time="1731001560.756464" Severity="10" LogGroup="default" Process="fdbmonitor": Started FoundationDB Process Monitor 7.3 (v7.3.55)
Time="1731001560.756531" Severity="10" LogGroup="default" Process="fdbmonitor": Watching conf file /etc/foundationdb/foundationdb.conf
Time="1731001560.756538" Severity="10" LogGroup="default" Process="fdbmonitor": Watching conf dir /etc/foundationdb/ (2)
Time="1731001560.756544" Severity="10" LogGroup="default" Process="fdbmonitor": Loading configuration /etc/foundationdb/foundationdb.conf
Time="1731001560.756629" Severity="10" LogGroup="default" Process="fdbmonitor": Starting backup_agent.1
Time="1731001560.756687" Severity="10" LogGroup="default" Process="fdbmonitor": Starting fdbserver.4500
Time="1731001560.756743" Severity="10" LogGroup="default" Process="backup_agent.1": Launching /usr/lib/foundationdb/backup_agent/backup_agent (2076108) for backup_agent.1
Time="1731001560.756780" Severity="10" LogGroup="default" Process="fdbserver.4500": Launching /usr/sbin/fdbserver (2076109) for fdbserver.4500
Time="1731001560.757485" Severity="40" LogGroup="default" Process="backup_agent.1": Unable to launch /usr/lib/foundationdb/backup_agent/backup_agent for backup_agent.1 (execv error 8: Exec format error)
Time="1731001560.757531" Severity="20" LogGroup="default" Process="backup_agent.1": Process 2076108 exited 0, restarting in 0 seconds
Time="1731001560.757633" Severity="10" LogGroup="default" Process="backup_agent.1": Launching /usr/lib/foundationdb/backup_agent/backup_agent (2076111) for backup_agent.1
Time="1731001560.758190" Severity="40" LogGroup="default" Process="backup_agent.1": Unable to launch /usr/lib/foundationdb/backup_agent/backup_agent for backup_agent.1 (execv error 8: Exec format error)
Time="1731001560.758217" Severity="20" LogGroup="default" Process="backup_agent.1": Process 2076111 exited 0, restarting in 59 seconds
Time="1731001560.761139" Severity="40" LogGroup="default" Process="fdbserver.4500": : Unsupported system page size
Time="1731001560.761145" Severity="40" LogGroup="default" Process="fdbserver.4500": : Unsupported system page size
Time="1731001560.761148" Severity="40" LogGroup="default" Process="fdbserver.4500": : Unsupported system page size
Time="1731001560.763954" Severity="40" LogGroup="default" Process="fdbserver.4500": : Unsupported system page size
Time="1731001560.763958" Severity="40" LogGroup="default" Process="fdbserver.4500": libc++abi: terminating due to uncaught exception of type St9bad_alloc: std::bad_alloc
Time="1731001561.836147" Severity="20" LogGroup="default" Process="fdbserver.4500": Process 2076109 terminated by signal 6, restarting in 0 seconds
Time="1731001561.836301" Severity="10" LogGroup="default" Process="fdbserver.4500": Launching /usr/sbin/fdbserver (2076127) for fdbserver.4500
Time="1731001561.836734" Severity="40" LogGroup="default" Process="fdbserver.4500": : Unsupported system page size
Time="1731001561.836739" Severity="40" LogGroup="default" Process="fdbserver.4500": : Unsupported system page size
Time="1731001561.836743" Severity="40" LogGroup="default" Process="fdbserver.4500": : Unsupported system page size
Time="1731001561.836769" Severity="40" LogGroup="default" Process="fdbserver.4500": : Unsupported system page size
Time="1731001561.836772" Severity="40" LogGroup="default" Process="fdbserver.4500": libc++abi:
Time="1731001561.836775" Severity="40" LogGroup="default" Process="fdbserver.4500": terminating due to uncaught exception of type St9bad_alloc: std::bad_alloc
```
This is a common problem on these machines, linux is running with 16kb pages for kernel and userland.
```
miguel@armadura vm % zgrep -i PAGE_SIZE /boot/config-$(uname -r)
CONFIG_HAVE_PAGE_SIZE_16KB=y
CONFIG_PAGE_SIZE_16KB=y
CONFIG_PAGE_SIZE_LESS_THAN_64KB=y
CONFIG_PAGE_SIZE_LESS_THAN_256KB=y
miguel@armadura vm % getconf PAGESIZE
16384
```
I have a local build of jemalloc w/ 16kb pages, but because fdb is built statically I can't use that.
```
miguel@armadura fdb % ldd fdbmonitor.aarch64
linux-vdso.so.1 (0x0000ffff8a3fc000)
libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffff8a2e0000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff8a110000)
/lib/ld-linux-aarch64.so.1 (0x0000ffff8a3b0000)
miguel@armadura fdb % ldd fdbserver.aarch64
linux-vdso.so.1 (0x0000ffff95598000)
liblzma.so.5 => /lib/aarch64-linux-gnu/liblzma.so.5 (0x0000ffff954f0000)
libz.so.1 => /lib/aarch64-linux-gnu/libz.so.1 (0x0000ffff954b0000)
libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffff95400000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff95230000)
/lib/ld-linux-aarch64.so.1 (0x0000ffff95560000)
```
```
miguel@armadura fdb % uname -a
Linux armadura 6.11.0-1001-asahi-arm #1-Ubuntu SMP PREEMPT_DYNAMIC Sat Sep 28 14:42:09 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
miguel@armadura fdb % lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04.1 LTS
Release: 24.04
Codename: noble
```
Contributor guide
Research direction
Start by reproducing the failure with the arm64 fdbmonitor and fdbserver binaries on a Linux system reporting 16 KB pages. Trace how the statically linked jemalloc is selected and built, then verify that the resulting binaries start without the unsupported-page-size or bad_alloc errors. No source files or tests are named in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux
- Domain
- build-system, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100