UCL-ARC / UCL-ARC/rcps-buildscripts

Julia wrapper to disable pkgimages on Kathleen not needed anymore

Open
#586 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
42
Forks
26
PR merge metrics
No merged PRs in 30d

Description

Last year I reported that on Kathleen lld (the LLVM linker) used to crash badly, providing the following reproducer (the linker invocation isn't quite complete and it wouldn't create a valid executable anyway, but normally it shouldn't crash with a bus error):

$ echo 'int main(void) { return 0; }' | gcc -x c -c -o main.o -
$ /shared/ucl/apps/llvm/8.0.0/gnu-4.9.2/bin/lld -flavor gnu -o main main.o
Stack dump:
0.      Program arguments: /shared/ucl/apps/llvm/8.0.0/gnu-4.9.2/bin/lld -flavor gnu -o main main.o
 #0 0x000000000050c35a llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/shared/ucl/apps/llvm/8.0.0/gnu-4.9.2/bin/lld+0x50c35a)
 #1 0x000000000050a5ba llvm::sys::RunSignalHandlers() (/shared/ucl/apps/llvm/8.0.0/gnu-4.9.2/bin/lld+0x50a5ba)
 #2 0x000000000050a6f1 SignalHandler(int) (/shared/ucl/apps/llvm/8.0.0/gnu-4.9.2/bin/lld+0x50a6f1)
 #3 0x00007fc633f4d630 __restore_rt (/lib64/libpthread.so.0+0xf630)
 #4 0x00000000006ca94a void lld::elf::writeResult<llvm::object::ELFType<(llvm::support::endianness)1, true> >() (/shared/ucl/apps/llvm/8.0.0/gnu-4.9.2/bin/lld+0x6ca94a)
 #5 0x00000000005ce5ad void lld::elf::LinkerDriver::link<llvm::object::ELFType<(llvm::support::endianness)1, true> >(llvm::opt::InputArgList&) (/shared/ucl/apps/llvm/8.0.0/gnu-4.9.2/bin/lld+0x5ce5ad)
 #6 0x000000000044acb2 lld::elf::LinkerDriver::main(llvm::ArrayRef<char const*>) (/shared/ucl/apps/llvm/8.0.0/gnu-4.9.2/bin/lld+0x44acb2)
 #7 0x00000000005cf5c0 lld::elf::link(llvm::ArrayRef<char const*>, bool, llvm::raw_ostream&) (/shared/ucl/apps/llvm/8.0.0/gnu-4.9.2/bin/lld+0x5cf5c0)
 #8 0x00000000004483cf main (/shared/ucl/apps/llvm/8.0.0/gnu-4.9.2/bin/lld+0x4483cf)
 #9 0x00007fc6327b9555 __libc_start_main (/lib64/libc.so.6+0x22555)
#10 0x00000000004f72ac _start (/shared/ucl/apps/llvm/8.0.0/gnu-4.9.2/bin/lld+0x4f72ac)
Bus error

This issue also affected julia, which in some cases needs to use lld under the hood, then Owain created this wrapper to workaround the issue (--pkgimages=no means that julia won't compile native code for user packages, thus not requiring the linker, but this also makes using packages in julia slower, as there's no cache of native code which has always to be recompiled on demand in all new sessions).

However, as far as I can tell, after the upgrade to the new filesystem on kathleen all these issues seem to be gone, and the Julia wrapper doesn't seem to be needed anymore.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the Julia wrapper introduced in commit 6b7c5d914a5028a09343b5e40040eb6f92b456c0 and reproduce the lld and Julia behavior on Kathleen after the filesystem upgrade. Confirm that Julia no longer needs --pkgimages=no, then remove the obsolete wrapper and verify normal package native-code caching works.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia, shell
Domain
hpc, tooling
Issue type
Refactor
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.