improve performance of pointer scan for leak reachability analysis

_From [derek.br...@gmail.com](https://code.google.com/u/117968039472581148324/) on December 10, 2010 17:58:04_

PR 475518

my initial implementation of scanning for pointers for reachability analysis produces a noticeable 1-second
delay at app exit (and at nudge time for PR 428709). this case covers trying to speed that up. it's not
identifying the defined region that's slow, it's the pointer scan itself.

adding to this case a FIXME from my code that may improve perf noticeably:
skipping regions that have been read-only since being loaded (.text,
.rodata, etc.). we'll need extra tracking since today we can't tell
whether they were made writable and then changed back to read-only.
we could also skip writable regions that have not been written to since
being loaded since they can't point into the heap. technically
the app could write a heap pointer to a file and then mmap it in but we'll
ignore that.

my initial imp is going to skip r-x regions. I'm not going to skip all non-writable
though. this case covers adding modified-since-load info to
both avoid false neg w/ r-x and to skip more non-writable.

it does take substantial time on large apps.
we should probably fix up -no_count_leaks and document it, and perhaps make it default so that
nudges can be used for just updating the error summary w/o waiting 8 minutes?
just turning it off today => bugs: maybe somebody else needs
op_record_allocs in alloc.c. we still record for pre_us.

for PR 485354 I have the leak scan on windows skipping read-only image regions: once this case checks for history that can be done for *nix too

Server: perforce-panda.eng.vmware.com:1985

PR 520916: leak-check-only mode without losing accuracy
- added -leaks_only option that does not do any shadowing
- added -shadowing option as an internal debugging method for
disabling everything except malloc instrumentation.
it could be more thorough (e.g., don't need known_table, etc.) but
should prove useful. this is what my original -leaks_only experiment
used so I kept the code under -no_shadowing when I added stack zeroing.
- split os_shared_{pre,post}_syscall() from os_shadow_{pre,post}_syscall()
for more than just options.shadowing: also for PR 536878 where
drheapstat will need fork-following
- to rule out beyond-TOS, needs xsp in each thread at exit time: for that
I needed PR 536058 in DR (diff sent earlier, included in this tree since
needed to test)
- to rule out stale stack entries, added zeroing on stack allocation by
adapting drmem's adjust_esp shadow code
- for locating live mallocs inside an unmapped heap arena, which is done
using shadow info for regular drmem, I'm brute-force looking up every
8 bytes: but only on Windows since I'm assuming glibc malloc never
unmaps an arena that contains live mallocs. xref PR 535568's attempt
to replace the malloc table w/ an interval tree: too expensive!
- added a -leaks-only test

PR 485354: # possible leaks nondet on nudge test on Windows
My leaks-only test hit PR 485354 so I investigated:
- it's coming from .pdata RVA entries
- I have windows scans skipping read-only image sections now:
PR 475518 covers doing that properly and on both plaforms by
monitoring history

The beyond-TOS + zero-on-stack-alloc seems to be working in practice as far
as I can tell (apps aren't exactly deterministic but results
seem to match pretty well w/ full shadowing). Zeroing is not 100%
transparent but it's close enough for me. We'll need more data on more
apps but I think it's going to work out, and the overhead should be a
strict subset of my original shadow-writes proposal. Unfortunately the
overhead is still significant: 2.2x on crafty (vs 1.6x for DrHeapstat).

xref PR 536878: add leak checking feature to Dr. Heapstat a la Dr. Memory
-leaks_only

future work under PR 539395: improve accuracy of -leaks_only:
- issues w/ ignoring beyond-TOS w/ alternate stacks:
- if currently on alt sigstack will look beyond TOS on main stack
- if currently on main stack will look at whole alt stack
could intercept signal + sigreturn.
- are there uninit fields in kernel stack allocs for signal/Ki?
- use an interval tree of mallocs (despite perf cost: see PR 535568) or
some other way to improve accuracy and efficiency of locating live
mallocs in an unmapped arena

_Original issue: http://code.google.com/p/drmemory/issues/detail?id=151_