client support for persistent and process-shared caches

_From [derek.br...@gmail.com](https://code.google.com/u/117968039472581148324/) on February 24, 2009 10:51:55_

this was PR 305329

We should explore pcache support now before the 0.9.6 release to determine
whether we can add it later while preserving backward compatibility.
I think we can:
- need client id => we already have it
need to store in pcache, and make part of pcache namespace for
simultaneous pcaches.
for multiple clients, put all ids in namespace, so won't load unless
exact same combination is present on subsequent run.

also add event callback when loading pcache => client can decide whether
to load based on whether already has desired instru in pcache. thus,
want client version # as well (else, client has to bump id#?). could
specify separately when do persist nudge and so not change the dr_init
signature, or make an event callback if DR needs to ask for it. for
multiple clients, if any one of them rejects it, don't use it.

problem: the client id is really just for the current instance of DR and is
specified by the user; it's so the client can request its options, etc.
it's not a persistent tool identifier.

so how will a client know whether its instrumentation is present in a
pcache? do we want two types of identifiers: one that's in the
namespace, for simultaneous pcaches, and another that's in the pcache
header that the client can check for versioning? what if for pcaches we
added two event callbacks: get_client_id() and get_client_version().
they're ints, or maybe 64-bit ints. we use the 1st in the pcache
namespace, and store the 2nd in the pcache header for retrieval when we
load. a client that doesn't supply them isn't allowed to persist. when
DR creates a pcache it will call these for each client so it can label
the pcache with which tools were active when it was created.
problem: confusing having this new notion of id vs the client_id_t used
in dr_init.

=> solution: use client library identifying hash (similar to what we use
for app modules for pcaches and hot patches: checksum + timestamp + size)
in the namespace. if the client is rebuilt, pcaches will have to be
re-generated: that shouldn't be a big deal, and can avoid bugs where the
client dev thinks a change shouldn't affect persistence but really it does.
- DR_EMIT_DO_NOT_PERSIST flag so can control on individual fragment
basis => addition only
- query whether ilist/fragment is persistable => addition only
- control over where pcaches are stored and who they are shared with;
also over when to persist
=> drdeploy/global options, plus maybe API call to freeze/persist
- relocation:
- app relocation: client non-meta code: if set translation fields right,
our own relocation support should cover it
- client meta code relocation: require client to use position-independent
code? else, how point client at its code later to fix up? make it
provide a relocation table? in any case should all be additions.

We'll want to finish some internal pieces first:
- PR 214016/9581, PR 214084/9649: relocation support
- PR 270739: [x64] need vendor pcache flag for lcall,ljmp rex.w differences
- PR 214155/9720: offline publisher + two-level hierarchy of sharing
- PR 215036/10601, PR 215277/10842: two-phase, lazy md5
- perf: PR 215260/10825: measure -no_persist_map_rw_separate prefetch impact
- perf: PR 210308/5836: don't want private bbs not being in ibl to
cause perf problems!
- PR 206574/2096 (xref PR 215247/10812 winlogon synchall issues)

We also have some performance issues (xref PR 326610):
- PR 213262: support -no_indirect_stubs w/ -coarse_units, to avoid losing perf
- PR 326815: [perf] -coarse_units slow on gcc and gap

_Original issue: http://code.google.com/p/dynamorio/issues/detail?id=40_