DynamoRIO / DynamoRIO/drmemory

CRASH running Chromium net_unittests: DR heap units are read-only from cache cons

Open
#1,820 3 comments 0 reactions 1 assignee Claimed by @derekbruening View on GitHub
Bug-ToolCrash Hotlist-Chrome Hotlist-Release OpSys-Windows
Dominant language
C
Stars
2.7k
Forks
290
PR merge metrics
No merged PRs in 30d

Description

I was trying to repro the V8 app crash in http://crbug.com/557993 locally by running a full shard of net_unittests, but my local run hit an internal crash:

```
[ RUN ] ProxyResolverV8TracingTest.InfiniteDNSSequence

~~Dr.M~~ WARNING: application exited with abnormal code 0xffffffff
```

It's in heap_vmareas_synch_units()

Xref the earlier crashes we never figured out: #1743. This is also a
write to a read-only page.
The theory there was that this is like #1354.
Indeed:

```
0:013> !vprot 0e501010
BaseAddress: 0e501000
AllocationBase: 0e500000
AllocationProtect: 00000001 PAGE_NOACCESS
RegionSize: 0003e000
State: 00001000 MEM_COMMIT
Protect: 00000002 PAGE_READONLY
Type: 00020000 MEM_PRIVATE

0:013> dt dynamorio!executable_areas
0x20ab7e3c
+0x000 buf : 0x2131abc0 vm_area_t
+0x004 size : 0n200
+0x008 length : 0n134
+0x00c flags : 1
+0x010 lock : _read_write_lock_t
+0x02c free_payload_func : (null)
+0x030 split_payload_func : (null)
+0x034 should_merge_func : (null)
+0x038 merge_payload_func : (null)
0:013> dt -ba134 vm_area_t @@(dynamorio!executable_areas->buf)
[2] @ 2131abe8
---------------------------------------------
+0x000 start : 0x0e501000 " ???"
+0x004 end : 0x0e53f000 "--- memory read error at address 0x0e53f000 ---"
+0x008 vm_flags : 0x801 <== VM_EXECUTED_FROM | VM_WRITABLE
+0x00c frag_flags : 0
+0x010 custom :
+0x000 frags : (null)
+0x000 client : (null)
```

Stack is not showing up properly in windbg:

```
0:013> kn 10
# ChildEBP RetAddr
00 266a9ee4 6b9d7a5c ntdll!NtRaiseHardError+0x12
01 266a9f28 6b9c3617 dynamorio!nt_messagebox+0x7c [d:\derek\dr\git\src\core\win32\ntdll.c @ 3771]
02 266a9f4c 6b965769 dynamorio!debugbox+0x77 [d:\derek\dr\git\src\core\win32\os.c @ 4910]
03 266a9f54 6c707041 dynamorio!notify+0xd9 [d:\derek\dr\git\src\core\utils.c @ 1963]
04 (Inline) -------- base!std::allocator,std::allocator > >::deallocate+0x1 [z:\derek\depot_tools\win_toolchain\vs2013_files\vc\include\xmemory0 @ 573]
05 (Inline) -------- base!std::_Wrap_alloc,std::allocator > > >::deallocate+0x1 [z:\derek\depot_tools\win_toolchain\vs2013_files\vc\include\xmemory0 @ 858]
06 (Inline) -------- base!std::vector,std::allocator >,std::allocator,std::allocator > > >::_Tidy+0x22 [z:\derek\depot_tools\win_toolchain\vs2013_files\vc\include\vector @ 1628]
07 (Inline) -------- base!std::vector,std::allocator >,std::allocator,std::allocator > > >::{dtor}+0x22 [z:\derek\depot_tools\win_toolchain\vs2013_files\vc\include\vector @ 945]
```

But I found the context:

```
0:013> .cxr 266aacc8
eax=0e501000 ebx=00001000 ecx=20a81024 edx=05620000 esi=0e715508 edi=20abc314
eip=6b96d364 esp=266ab12c ebp=266ab1b0 iopl=0 nv up ei pl nz na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206
dynamorio!heap_vmareas_synch_units+0x54:
6b96d364 c6401000 mov byte ptr [eax+10h],0 ds:002b:0e501010=01
0:013> kn
*** Stack trace for last set context - .thread/.cxr resets it
# ChildEBP RetAddr
00 266ab140 6b976d5b dynamorio!heap_vmareas_synch_units+0x54 [d:\derek\dr\git\src\core\heap.c @ 2547]
01 266ab148 6b97725e dynamorio!dynamo_vm_areas_start_reading+0x8b [d:\derek\dr\git\src\core\vmareas.c @ 3561]
02 266ab154 6b9e856f dynamorio!is_dynamo_address+0x1e [d:\derek\dr\git\src\core\vmareas.c @ 3696]
03 266ab164 00b098cd dynamorio!redirect_RtlFreeHeap+0x3f [d:\derek\dr\git\src\core\win32\drwinapi\ntdll_redir.c @ 398]
04 266ab1b0 6b82e52e msvcrt_b00000!free+0xcd
WARNING: Stack unwind information not available. Following frames may be wrong.
05 266ab1bc 6b8cbd8d dbghelp_6b800000!SymGetFileLineOffsets64+0x1ae
06 266ab1d4 6b8cc0f6 dbghelp_6b800000!MiniDumpReadDumpStream+0x6e73d
07 266ab1e8 6b8c8c21 dbghelp_6b800000!MiniDumpReadDumpStream+0x6eaa6
08 266ab1fc 6b8d3c01 dbghelp_6b800000!MiniDumpReadDumpStream+0x6b5d1
09 266ab218 6b8d46cc dbghelp_6b800000!MiniDumpReadDumpStream+0x765b1
0a 266ab268 6b8c8abe dbghelp_6b800000!MiniDumpReadDumpStream+0x7707c
0b 266ab280 6b8c80f0 dbghelp_6b800000!MiniDumpReadDumpStream+0x6b46e
0c 266ab298 6b8ca397 dbghelp_6b800000!MiniDumpReadDumpStream+0x6aaa0
0d 266ab2e4 6b88ef6b dbghelp_6b800000!MiniDumpReadDumpStream+0x6cd47
0e 266ab314 6b819004 dbghelp_6b800000!MiniDumpReadDumpStream+0x3191b
0f 266ab348 6b819a9e dbghelp_6b800000+0x19004
10 266ac028 6b81a936 dbghelp_6b800000+0x19a9e
11 266ac2a4 6b842bb4 dbghelp_6b800000+0x1a936
12 266ac4c8 6b840fe5 dbghelp_6b800000!ImagehlpApiVersionEx+0x2174
13 266ac970 6b8415e5 dbghelp_6b800000!ImagehlpApiVersionEx+0x5a5
14 266acdf0 6b8394c5 dbghelp_6b800000!ImagehlpApiVersionEx+0xba5
15 266ace58 73ac7ed0 dbghelp_6b800000!SymLoadModuleExW+0x65
16 266ad0e0 73ac7c3a drmemorylib!load_module+0x240 [d:\derek\dr\git\src\ext\drsyms\drsyms_windows.c @ 396]
17 266ad0f8 73ac78c0 drmemorylib!lookup_or_load+0x7a [d:\derek\dr\git\src\ext\drsyms\drsyms_windows.c @ 459]
18 266ad148 73ac786a drmemorylib!drsym_lookup_address_local+0x50 [d:\derek\dr\git\src\ext\drsyms\drsyms_windows.c @ 662]
19 266ad160 739e1ca3 drmemorylib!drsym_lookup_address+0x2a [d:\derek\dr\git\src\ext\drsyms\drsyms_windows.c @ 1493]
1a 266ad438 739f0184 drmemorylib!lookup_func_and_line+0x473 [d:\derek\drmemory\git\src\common\callstack.c @ 533]
1b 266ad5cc 739f2da6 drmemorylib!packed_frame_to_symbolized+0x17a4 [d:\derek\drmemory\git\src\common\callstack.c @ 2062]
1c 266ad648 73a275f8 drmemorylib!packed_callstack_to_symbolized+0x7a6 [d:\derek\drmemory\git\src\common\callstack.c @ 2106]
1d 266ad978 73a260ca drmemorylib!report_error+0x1498 [d:\derek\drmemory\git\src\drmemory\report.c @ 2862]
1e 266adbd4 73859ef4 drmemorylib!report_unaddressable_access+0xe1a [d:\derek\drmemory\git\src\drmemory\report.c @ 3158]
1f 266ae3d4 7385eb64 drmemorylib!handle_mem_ref_internal+0x5584 [d:\derek\drmemory\git\src\drmemory\readwrite.c @ 4125]
20 266ae650 7383f457 drmemorylib!check_mem_opnd+0x2cc4 [d:\derek\drmemory\git\src\drmemory\readwrite.c @ 4297]
21 266aed08 73851dd3 drmemorylib!slow_path_with_mc+0x57b7 [d:\derek\drmemory\git\src\drmemory\readwrite.c @ 3215]
22 266aeeb0 2133003a drmemorylib!slow_path+0x63 [d:\derek\drmemory\git\src\drmemory\readwrite.c @ 3289]
23 266aeeec 6b96f2f0 0x2133003a
24 266aeefc 6bca07c4 dynamorio!heap_free+0x40 [d:\derek\dr\git\src\core\heap.c @ 3773]
25 (Inline) -------- v8!v8::internal::TemplateHashMapImpl::Initialize+0x33 [z:\derek\chromium\src\v8\src\hashmap.h @ 274]
26 (Inline) -------- v8!v8::internal::TemplateHashMapImpl::{ctor}+0x3a [z:\derek\chromium\src\v8\src\hashmap.h @ 104]
27 266aef30 00000000 v8!v8::internal::ObjectLiteral::CalculateEmitStore+0x44 [z:\derek\chromium\src\v8\src\ast.cc @ 365]
```

Code is here:

```
if (dynamo_areas_pending_remove) {
dynamo_areas_pending_remove = false;
remove_dynamo_heap_areas();
/* When heap units are removed from the dynamo_area, they should be
* marked so. See case 4196.
*/
for (u = heapmgt->heap.units; u != NULL; u = u->next_global)
u->in_vmarea_list = false;
```

```
0:013> dt -b dynamorio!heapmgt
0x20a81024
+0x000 vmheap : vm_heap_t
+0x000 start_addr : 0x20a80000 "--- memory read error at address 0x20a80000 ---"
+0x004 end_addr : 0x30a80000 "--- memory read error at address 0x30a80000 ---"
+0x420 heap : _heap_t
+0x000 units : 0x0eb51000
+0x004 dead : (null)
+0x008 num_dead : 0
+0x42c global_units : _thread_units_t
+0x000 top_unit : 0x20a81000
+0x004 cur_unit : 0x0e9f1000

!list -t dynamorio!_heap_unit_t.next_local -x "dt" -a "dynamorio!_heap_unit_t" @@(dynamorio!heapmgt->global_units.top_unit)

=>
dynamorio!_heap_unit_t
+0x000 start_pc : 0x0e501020 ""
+0x004 end_pc : 0x0e53f000 "--- memory read error at address 0x0e53f000 ---"
+0x008 cur_pc : 0x0e53f000 "--- memory read error at address 0x0e53f000 ---"
+0x00c reserved_end_pc : 0x0e53f000 "--- memory read error at address 0x0e53f000 ---"
+0x010 in_vmarea_list : 1 ''
+0x014 next_local : 0x0e541000 _heap_unit_t
+0x018 next_global : 0x0e1a1000 _heap_unit_t
+0x01c prev_global : 0x0e541000 _heap_unit_t
```

So is this like #1354 except here the app code is in a DR heap unit
instead of a replace_malloc raw mmap? But how did the app get code into a
DR heap unit?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.