llvm / llvm/llvm-project

Out of memory trying to link AllClangTests binary using ld.lld on 32-bit Arm Linux

Open
#172,659 4 comments 0 reactions 0 assignees View on GitHub
build-problem
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

Linaro has a bot armv8-lld-2stage (https://lab.llvm.org/buildbot/#/builders/122) where we built llvm and clang on 32-bit Armv8 Linux. Using lld being the key thing we want to validate.

Since https://github.com/llvm/llvm-project/pull/134196 we have sometimes had this error, and a lot more frequently recently:
```
FAILED: tools/clang/unittests/AllClangUnitTests
: && /usr/local/bin/c++ -mcpu=neoverse-n1 -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wno-pass-failed -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG -fuse-ld=lld -Wl,--color-diagnostics -Wl,--gc-sections tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Lex/DependencyDirectivesScannerTest.cpp.o <...>tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/AllClangUnitTests.cpp.o -o tools/clang/unittests/AllClangUnitTests -Wl,-rpath,/home/tcwg-buildbot/worker/clang-armv8-lld-2stage/stage1/lib lib/libLLVMSupport.a lib/libLLVMFrontendHLSL.a lib/libLLVMSupport.a lib/libLLVMAArch64CodeGen.a <...> lib/libLLVMDemangle.a && :
ld.lld: error: failed to open tools/clang/unittests/AllClangUnitTests: Cannot allocate memory
```
(https://lab.llvm.org/buildbot/#/builders/122/builds/2509 is one build)

I've measured a few builds and ld.lld consistently reaches the 3 point something GB of RAM limit per process.

ld does the link with a peak of ~1.5GB, but using that would be against the point of the bot.

I have tried various options to reduce lld's memory usage, but could not reduce it enough:
* We have threading disabled for the built lld, and I set threads to 1 for the 1st stage lld.
* `--no-mmap-output-file`
* `-Wl,--build-id=none -Wl,--no-relax` (a shot in the dark thinking it might reduce the binary size)
* `-O0`

I need to do a survey of all of lld options to find more.

The alternative is to provide a way for builds to opt back into the per-unit-test binary that we had before. I'll post a draft change that does that for reference purposes.

Contributor guide

Open the contributing guide

Research direction

Start with the armv8-lld-2stage buildbot failure and the shown ld.lld link command for tools/clang/unittests/AllClangUnitTests. Survey the attempted options, including --no-mmap-output-file, --build-id=none, --no-relax, and -O0, and compare lld's memory use with ld. Done means identifying a workable way to complete the link or documenting the alternative of restoring per-unit-test binaries.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, linux
Domain
build-system, compilers
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.