Out of memory trying to link AllClangTests binary using ld.lld on 32-bit Arm Linux
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
Linaro has a bot armv8-lld-2stage (https://lab.llvm.org/buildbot/#/builders/122) where we built llvm and clang on 32-bit Armv8 Linux. Using lld being the key thing we want to validate.
Since https://github.com/llvm/llvm-project/pull/134196 we have sometimes had this error, and a lot more frequently recently:
```
FAILED: tools/clang/unittests/AllClangUnitTests
: && /usr/local/bin/c++ -mcpu=neoverse-n1 -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wno-pass-failed -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG -fuse-ld=lld -Wl,--color-diagnostics -Wl,--gc-sections tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Lex/DependencyDirectivesScannerTest.cpp.o <...>tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/AllClangUnitTests.cpp.o -o tools/clang/unittests/AllClangUnitTests -Wl,-rpath,/home/tcwg-buildbot/worker/clang-armv8-lld-2stage/stage1/lib lib/libLLVMSupport.a lib/libLLVMFrontendHLSL.a lib/libLLVMSupport.a lib/libLLVMAArch64CodeGen.a <...> lib/libLLVMDemangle.a && :
ld.lld: error: failed to open tools/clang/unittests/AllClangUnitTests: Cannot allocate memory
```
(https://lab.llvm.org/buildbot/#/builders/122/builds/2509 is one build)
I've measured a few builds and ld.lld consistently reaches the 3 point something GB of RAM limit per process.
ld does the link with a peak of ~1.5GB, but using that would be against the point of the bot.
I have tried various options to reduce lld's memory usage, but could not reduce it enough:
* We have threading disabled for the built lld, and I set threads to 1 for the 1st stage lld.
* `--no-mmap-output-file`
* `-Wl,--build-id=none -Wl,--no-relax` (a shot in the dark thinking it might reduce the binary size)
* `-O0`
I need to do a survey of all of lld options to find more.
The alternative is to provide a way for builds to opt back into the per-unit-test binary that we had before. I'll post a draft change that does that for reference purposes.
Contributor guide
Research direction
Start with the armv8-lld-2stage buildbot failure and the shown ld.lld link command for tools/clang/unittests/AllClangUnitTests. Survey the attempted options, including --no-mmap-output-file, --build-id=none, --no-relax, and -O0, and compare lld's memory use with ld. Done means identifying a workable way to complete the link or documenting the alternative of restoring per-unit-test binaries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux
- Domain
- build-system, compilers
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100