DynamoRIO / DynamoRIO/dynamorio
ASSERT_CURIOSITY "crashed while walking dynamic header" on AArch64
- Dominant language
- C
- Stars
- 3.2k
- Forks
- 629
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 31
Description
This issue has been raised as a consequence of https://groups.google.com/forum/#!topic/dynamorio-users/UTWiYoc9TvA
The library load which exposes this failure in the ELF loader is libarmflang.so shipped as part of the Arm HPC compiler 19.0. I think it may be related to https://github.com/DynamoRIO/dynamorio/issues/1589 as the library is quite large (5.2M).
A SIGBUS (not SIGSEGV!) is caused by the strlen() call in the soname check in core/unix/module_elf.c:
```
/* test string readability while still in try/except
* in case we screwed up somewhere or module is
* malformed/only partially mapped */
if (*soname != NULL && strlen(*soname) == -1) {
ASSERT_NOT_REACHED();
}
```
It was reproduced using a simple Fortran test case built with armflang:
```
$ cat hw.f
program hello
print *, "Hello World!"
end program hello
$
$ armflang hw.f -o hw
$
$ ./hw
Hello World!
$
```
ldd shows which libraries are linked:
```
$ ldd hw
linux-vdso.so.1 (0x0000ffffb0649000)
libarmflang.so => /opt/arm/arm-hpc-compiler-19.0_Generic-AArch64_SUSE-12_aarch64-linux/lib/libarmflang.so (0x0000ffffb0243000)
libomp.so => /opt/arm/arm-hpc-compiler-19.0_Generic-AArch64_SUSE-12_aarch64-linux/lib/libomp.so (0x0000ffffb0165000)
libm.so.6 => /lib64/libm.so.6 (0x0000ffffb0092000)
librt.so.1 => /lib64/librt.so.1 (0x0000ffffb0071000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000ffffb003c000)
libc.so.6 => /lib64/libc.so.6 (0x0000ffffafec6000)
libstdc++.so.6 => /opt/arm/gcc-8.2.0_Generic-AArch64_SUSE-12_aarch64-linux/lib64/libstdc++.so.6 (0x0000ffffafccd000)
libgcc_s.so.1 => /opt/arm/gcc-8.2.0_Generic-AArch64_SUSE-12_aarch64-linux/lib64/libgcc_s.so.1 (0x0000ffffafc9c000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000ffffafc7b000)
/lib/ld-linux-aarch64.so.1 (0x0000ffffb061b000)
$
```
The size of each library:
```
5.1M /opt/arm/arm-hpc-compiler-19.0_Generic-AArch64_SUSE-12_aarch64-linux/lib/libarmflang.so
1.1M /opt/arm/arm-hpc-compiler-19.0_Generic-AArch64_SUSE-12_aarch64-linux/lib/libomp.so
4.0K /lib64/libm.so.6
4.0K /lib64/librt.so.1
4.0K /lib64/libpthread.so.0
4.0K /lib64/libc.so.6
4.0K /opt/arm/gcc-8.2.0_Generic-AArch64_SUSE-12_aarch64-linux/lib64/libstdc++.so.6
844K /opt/arm/gcc-8.2.0_Generic-AArch64_SUSE-12_aarch64-linux/lib64/libgcc_s.so.1
4.0K /lib64/libdl.so.2
```
Adding a SYSLOG_INTERNAL_INFO() call to look at *soname for all libraries loaded by hw shows a corrupt soname for libarmflang:
```
$ git diff
diff --git a/core/unix/module_elf.c b/core/unix/module_elf.c
index 8d4db788..e2efca55 100644
--- a/core/unix/module_elf.c
+++ b/core/unix/module_elf.c
@@ -454,6 +454,8 @@ module_fill_os_data(ELF_PROGRAM_HEADER_TYPE *prog_hdr, /* PT_DYNAMIC entry */
/* test string readability while still in try/except
* in case we screwed up somewhere or module is
* malformed/only partially mapped */
+ SYSLOG_INTERNAL_INFO("DEBUG accessing *soname=[%p]", *soname);
+ SYSLOG_INTERNAL_INFO("DEBUG accessing *soname=[%s]\n", *soname);
if (*soname != NULL && strlen(*soname) == -1) {
ASSERT_NOT_REACHED();
}
lines 1-13/13 (END)
$ drrun -- ./hw
^E^KJ^E^U^FK^E^M^FJ^E^S^FK^E^K^Fº^D^K^E] <-- should be libarmflang.so
>
<(1+x) Handling our fault in a TRY at 0x00000000712e9274>
Hello World!
```
Interestingly, when run with GDB, a corrupt string appears in the SYSLOG_INTERNAL_INFO() output but not when looking at *soname pointer after SIGBUS:
```
. . .
Missing separate debuginfo for /lib/ld-linux-aarch64.so.1
Try: zypper install -C "debuginfo(build-id)=e8104675ba94d7c698d02558d81423b4fe5bff11"
Missing separate debuginfo for /lib64/libc.so.6
Try: zypper install -C "debuginfo(build-id)=9a94d7a23a8802dd0f216f3a3cea0eb29d703de0"
process 29487 is executing new program: /path/to/dynamorio/build/lib64/debug/libdynamorio.so
Program received signal SIGBUS, Bus error.
safe_read_asm_pre () at /path/to/dynamorio/core/arch/aarch64/aarch64.asm:400
400 strb w3, [ARG1]
(gdb)
0xffffbf6ce929 : "ld-linux-aarch64.so.1"
(gdb) x /bs 0x0000ffffbf6a4d81
0xffffbf6a4d81: "libarmflang.so" <-- valid soname !
(gdb)
```
With a development version of libarmflang.so (not yet released) which is bigger than the released version GDB says *soname hasn't been mapped:
```
$ gdb drrun
GNU gdb (GDB; openSUSE Tumbleweed) 8.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-suse-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /path/to/dynamorio/build/bin64/drrun...Reading symbols from /path/to/dynamorio/build/bin64/drrun.debug...done.
done.
(gdb) set confirm off
(gdb) add-symbol-file '/path/to/dynamorio/build/lib64/debug/libdynamorio.so' 0x0000000071013560
add symbol table from file "/path/to/dynamorio/build/lib64/debug/libdynamorio.so" at
.text_addr = 0x71013560
Reading symbols from /path/to/dynamorio/build/lib64/debug/libdynamorio.so...Reading symbols from /path/to/dynamorio/build/lib64/debug/libdynamorio.so.debug...done.
done.
(gdb) r ./hw
Starting program: /path/to/dynamorio/build/bin64/drrun ./hw
Missing separate debuginfo for /lib/ld-linux-aarch64.so.1
Try: zypper install -C "debuginfo(build-id)=e8104675ba94d7c698d02558d81423b4fe5bff11"
Missing separate debuginfo for /lib64/libc.so.6
Try: zypper install -C "debuginfo(build-id)=9a94d7a23a8802dd0f216f3a3cea0eb29d703de0"
process 29516 is executing new program: /path/to/dynamorio/build/lib64/debug/libdynamorio.so
Program received signal SIGBUS, Bus error.
0x00000000711cbe54 in our_vsnprintf (s=0x48ad5f61 "0x0000ffffbf6a5adf]", max=2048, fmt=0x713b99d8 "DEBUG accessing *soname=[%s]\n", ap=...)
at /path/to/dynamorio/core/iox.h:685
685 while (*str) {
(gdb) x /bs 0x0000ffffbf6ce929
0xffffbf6ce929 : "ld-linux-aarch64.so.1"
(gdb) x /bs 0x0000ffffbf6a5adf
0xffffbf6a5adf: <-- Not mapped!?
(gdb)
```
AFAICT there's nothing wrong with the arithmetic of *dynstr and soname_index, but I could be wrong.
However, the fact that the failure is a SIGBUS rather than SIGSEGV suggests that it's an alignment, cross page mapping or memory map sync error.
Some of the comments in module_elf.c imply that the relevant mapping(s) may not be in memory at the time soname is accessed.
Running with strace shows libarmflang.so is mapped to ffff99d94000-ffff9a16c000 with soname pointing to 0x0000ffff9a16ad81:
```
. . .
29725 mmap(NULL, 4026480, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0xffff99d94000
29725 openat(AT_FDCWD, "/proc/29725/maps", O_RDONLY) = 5
29725 read(5, "
00400000-00401000 r-xp 00000000 00:37 3893600 /path/to/hw
00401000-0041f000 ---p 00000000 00:00 0
0041f000-00421000 rw-p 0000f000 00:37 3893600 /path/to/hw
00421000-00422000 ---p 00000000 00:00 0
00422000-00423000 rw-p 00000000 00:00 0
4f1fc000-4f1fd000 ---p 00000000 00:00 0
4f1fd000-4f203000 rw-p 00000000 00:00 0
4f203000-4f205000 ---p 00000000 00:00 0
4f205000-4f213000 rw-p 00000000 00:00 0
. . .
4f2c3000-4f2d6000 ---p 00000000 00:00 0
4f2d6000-4f2d7000 rwxp 00000000 00:00 0
4f2d7000-4f2e6000 ---p 00000000 00:00 0
4f2e6000-4f2e9000 rwxp 00000000 00:00 0
4f2e9000-5f1fc000 ---p 00000000 00:00 0
71000000-713bc000 r-xp 00000000 00:37 3894297 /path/to/dynamorio/build/lib64/debug/libdynamorio.so
713bc000-713db000 ---p 00000000 00:00 0
713db000-713fe000 rw-p 003cb000 00:37 3894297 /path/to/dynamorio/build/lib64/debug/libdynamorio.so
713fe000-71434000 rw-p 00000000 00:00 0
71434000-71435000 ---p 00000000 00:00 0
ffff99d94000-ffff9a16c000 r-xp 00000000 00:39 1315912 /opt/arm/arm-hpc-compiler-19.0_Generic-AArch64_SUSE-12_aarch64-linux/lib/libarmflang.so
ffff9a16c000-ffff9a16d000 r--p 00000000 00:00 0 [vvar]
ffff9a16d000-ffff9a16e000 r-xp 00000000 00:00 0 [vdso]
ffff9a192000-ffff9a194000 rw-p 00000000 00:00 0
ffff9a194000-ffff9a1b3000 r-xp 00000000 00:2c 4322882 /lib64/ld-2.27.so
ffff9a1b3000-ffff9a1c3000 ---p 00000000 00:00 0
ffff9a1c3000-ffff9a1c5000 rw-p 0001f000 00:2c 4322882 /lib64/ld-2.27.so
ffff9a1c5000-ffff9a1c6000 rw-p 00000000 00:00 0
ffff9a1c6000-ffff9a1c7000 ---p 00000000 00:00 0
ffffca2d1000-ffffca2f3000 rw-p 00000000 00:00 0 [stack]
", 4103) = 3155
29725 close(5) = 0
29725 mprotect(0x4f25e000, 4096, PROT_READ|PROT_WRITE) = 0
29725 rt_sigprocmask(SIG_SETMASK, NULL, [], 8) = 0
29725 write(2, "\n", 47) = 47
29725 write(2, "\n", 56) = 56
. . .
```
Contributor guide
Assessment
This issue has not been assessed yet.