llvm / llvm/llvm-project

glibc longjmp_xor_key / InitializeLongjmpXorKey / UnmangleLongJmpSp about to break on most platforms

Open
#216,004 0 comments 0 reactions 0 assignees View on GitHub
compiler-rt:tsan platform:linux
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

glibc has just changed the implementation of longjmp_xor_key, so the current TSAN implementation appears to be about to break:

https://sourceware.org/git/?p=glibc.git;a=commit;h=78f1f0e39cd41d28ae771eb3498bc33780c85cfd

```
Consolidate the C pointer guard and align the assembly implementations

The per-architecture pointer_guard.h files all implemented some variant
of C PTR_MANGLE/PTR_DEMANGLE: the i386 and x86_64 versions did an
exclusive-or by the guard followed by a rotate left by
2 * sizeof (uintptr_t) + 1 bits (9 on 32-bit, 17 on 64-bit), while the
other targets did the exclusive-or only. The assembly macros likewise
rotated on i386 and x86_64 but not on the other targets.

Consolidate everything to use XOR + rotate uniformly: collapse the
per-architecture C headers into a single generic implementation, and
add the matching rotate to the assembly implementations that lacked it.

Targets with an assembly rotate instruction (aarch64, arm, loongarch,
powerpc, s390, csky) just add it. Those without one (alpha, sparc, sh)
synthesize the rotation: alpha and sparc with a shift/shift/or sequence,
which needs an extra scratch register added to the PTR_MANGLE/PTR_DEMANGLE
macros, sh with single-bit rotates, which needs no scratch. The s390
__longjmp and ____longjmp_chk demangle the return address and stack
pointer in C, so update those too.

Checked with setjmp, ptrguard and longjmp_chk tests for all supported
ABIs that can run under qemu (all but csky), and with builds for all
supported ABIs.

Reviewed-by: DJ Delorie
```

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked glibc commit, then search the TSAN implementation for longjmp_xor_key, InitializeLongjmpXorKey, and UnmangleLongJmpSp. Compare the current handling with glibc's uniform XOR-plus-rotate behavior across supported platforms; done means the affected longjmp handling remains compatible after the glibc change.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.