DynamoRIO / DynamoRIO/dynamorio
CRASH and APP CRASH on large ARM apps: regression
- Dominant language
- C
- Stars
- 3.2k
- Forks
- 629
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 30
Description
Despite chrome running fine all last week, after updating to the last few commits it keeps crashing. At first I thought this was a longstanding stability issue that happened to be surfacing now so I immediately debugged in case it was nondet (really miss debugger-initiated core dumps from Windows):
```
> bin32/drrun -stderr_mask 15 -msgbox_mask 12 -pause_via_loop -- ~/remote/out_arm/Release/chrome --no-sandbox --user-data-dir=/tmp/chrome
[5335:5335:0601/225139:ERROR:browser_main_loop.cc(172)] Running without the SUID sandbox! See https://code.google.com/p/chromium/wiki/LinuxSUIDSandboxDevelopment for more information on developing with the sandbox on.
(gdb) x/4i 0xb0099230
0xb0099230 : str.w r3, [r6], #4
0xb0099234 : beq.n 0xb0099244
(gdb) info line * 0xb0099230
Line 2887 of "/home/derek/dr/git/src/core/arch/arm/encode.c" starts at address 0xb009922e
...
(gdb) bt
#0 0xb0099230 in instr_encode_arch (dcontext=, instr=0x4fdbee44, copy_pc=0x0, final_pc=,
check_reachable=true, has_instr_opnds=0x0) at /home/derek/dr/git/src/core/arch/arm/encode.c:2887
#1 0xb00948ac in instr_encode_to_copy (dcontext=, instr=, copy_pc=,
final_pc=) at /home/derek/dr/git/src/core/arch/encode_shared.c:120
#2 0xb0059eb8 in set_linkstub_fields (num_indirect_stubs=0, num_direct_stubs=1, ilist=0x4fd4ed84, f=0x50061e84, dcontext=0x4fd07660,
emit=) at /home/derek/dr/git/src/core/emit.c:374
#3 emit_fragment_common (dcontext=0x4fd07660,
tag=0x4fd35004 "\370S\364\266\315j\245\265@\"\202\261\361\250\245\265`v\320O!\331\321O\201\331\321O\313\331\321O!\332\321Ok\332\321O\301\332\321O\v\333\321Oa\333\321O\253\333\321O\001\334\321OK\334\321O\241\334\321O\353\334\321O\377\017", ilist=0x4fd4ed84,
flags=16777216, vmlist=0x4ffa21e4, link_fragment=true, add_to_htable=true, replace_fragment=0x0)
at /home/derek/dr/git/src/core/emit.c:666
#4 0xb005a9dc in emit_fragment_ex (dcontext=, tag=, ilist=, flags=,
vmlist=0x4ffa21e4, link=true, visible=true) at /home/derek/dr/git/src/core/emit.c:999
#5 0xb00a02a8 in build_basic_block_fragment (dcontext=0x4fd07660,
start=0x4fd35004 "\370S\364\266\315j\245\265@\"\202\261\361\250\245\265`v\320O!\331\321O\201\331\321O\313\331\321O!\332\321Ok\332\321O\301\332\321O\v\333\321Oa\333\321O\253\333\321O\001\334\321OK\334\321O\241\334\321O\353\334\321O\377\017",
initial_flags=, link=, visible=true, for_trace=false, unmangled_ilist=0x0)
at /home/derek/dr/git/src/core/arch/interp.c:5072
#6 0xb0058852 in dispatch (dcontext=0x4fd07660) at /home/derek/dr/git/src/core/dispatch.c:205
(gdb) p *f
$2 = {
tag = 0x4fd35004 "\370S\364\266\315j\245\265@\"\202\261\361\250\245\265`v\320O!\331\321O\201\331\321O\313\331\321O!\332\321Ok\332\321O\301\332\321O\v\333\321Oa\333\321O\253\333\321O\001\334\321OK\334\321O\241\334\321O\353\334\321O\377\017",
flags = 16777216,
size = 32,
prefix_size = 4 '\004',
fcache_extra = 4 '\004',
start_pc = 0x50054ef4 "",
dcontext:
next_tag = 0x4fd35004
isa_mode = DR_ISA_ARM_A32,
last_exit = 0x50061e6c,
(gdb) x/30i 0x4fd35004
0x4fd35004: ; instruction: 0xb6f453f8
0x4fd35008: strlt r6, [r5, #2765]! ; 0xacd
0x4fd3500c: orrlt r2, r2, r0, asr #4
0x4fd35010: strlt r10, [r5, #2289]! ; 0x8f1
0x4fd35014: svcmi 0x00d07660
0x4fd35018: svcmi 0x00d1d921
not in a library. this is a gpu process.
(gdb) x/10i 0x50054ef4
0x50054ef4: ldr r0, [r10]
0x50054ef8: andeq r0, r0, r0
0x50054efc: andeq r0, r0, r0
instr it's trying to encode is a branch: OP_b (17).
1st instr is invalid:
(gdb) p *ilist
$5 = {
first = 0x4fdb7bc4,
last = 0x4fdbee44,
flags = 0,
translation_target = 0x0,
fall_through_bb = 0x0
}
(gdb) p ilist->first->opcode
$6 = 0
(gdb) p ilist->first->next->opcode
$7 = 17
(gdb) p ilist->first->next
$8 = (instr_t *) 0x4fdbee44
prior:
(gdb) p /x *(fragment_t*)((byte *)dcontext->last_exit - sizeof(fragment_t))
$13 = {
tag = 0x4fd35000,
flags = 0x1000030,
size = 0x20,
prefix_size = 0x4,
fcache_extra = 0x4,
start_pc = 0x50054ed0,
(gdb) x/10i 0x4fd35000
0x4fd35000: orrlt r2, r2, r0, asr #4
0x4fd35004: ; instruction: 0xb6f453f8
```
Taking a step back we see that many things crash and this is definitely a regression.
Crashing here too:
```
> bin32/drrun -no_enable_reset -stderr_mask 15 -- ~/remote/out_arm/Release/net_unittests --gtest_list_tests
Segmentation fault (core dumped)
```
This works:
```
> bin32/drrun -debug -no_enable_reset -stderr_mask 15 -- ~/chromium/out_arm/Release/base_unittests --gtest_list_tests
```
Crashes:
```
> bin32/drrun -debug -checklevel 0 -stderr_mask 15 -- ~/remote/out_arm/Release/net_unittests --gtest_filter=BitmapTest.Basics
<(1+x) Handling our fault in a TRY at 0x71230578>
0x2dbf64f2
>
Segmentation fault (core dumped)
```
binary search =>
culprit is addce83cf023867e10f2dc1bb1fb908b41971e66
"i#1662 ARM perf: optimize scratch reg pick for mangling"
I'm going to revert it to fix TOT.
In addition to fixing before re-landing, we should consider:
1) Investigate further this DR crash on encoding for an app with an invalid instr on ARM: add more tests of that
2) Add larger tests, or in particular targeted tests for whatever the specific bug turns out to be here (did not track down the precise bug but presumably it involves messing up the ibl targets or sthg)
Contributor guide
Assessment
This issue has not been assessed yet.