rare interruptions between instrumentation commit point and app instr execution cause tool inaccuracies

Most tools execute their added instrumentation prior to the execution of
the application instruction being observed. If an interruption arrives in
between the instrumentation and app execution, the tool can think the app
instruction did execute, and when control resumes at the same execution it
can double-count the instruction or mess up its state or whatever its
instrumentation focus happens to be. This is a general issue with the
instrumentation and its app instruction not being one atomic unit.

For always-asynchronous signals, DR always delays delivery until after a
block finishes executing. And for synchronous signals, the interrupted
instruction is re-executed, so this problem never shows up for "normal"
signals. It could happen for a not-normally-asynchronous signal sent
asynchronously: e.g., a SIGSEGV sent via SYS_kill. (Some parts of DR's
signal code look for is_sys_kill() but whether to deliver now does not,
today.)

This can also happen with DR relocating a thread: if its SIGUSR2 arrives
in between instru and app. Here, there the client can refuse to relocate.

The same interruptions could happen at any point during a multi-instruction
instrumentation sequence and not just right in between the end of the
instrumentation and the app instr. For some clients this is fine since
they don't "commit" their observation until the final couple of
instructions: e.g., a tracing tool not updating its buffer pointer until
the end.

For the signal case: DR still needs to call the client's restore_state
event. The client could conceivably roll back its instru actions, if it
can tell whether it already executed them. It could decode the raw cache
instructions to figure this out though this gets hacky and not always
foolproof; or if we implement i#3801 it could look at its own IR metadata.

The action items here are to document this more clearly, and perhaps
implement handling in our provided tool restore_state events? This adds
complexity though. We could look into whether we could disallow this from
ever happening: add the is_sys_kill() check (which for modern Linux kernels
should be solid) to record_pending_signal(), and for relocation we can have
DR detect this and re-try -- though that may lead to a lot of re-tries for
heavy-instrumentation clients; most signals are likely to arrive in the
middle of instrumentation.

Start by reading the signal-handling path around record_pending_signal() and the existing is_sys_kill() checks, then inspect how restore_state events are handled for instrumented code. Compare the signal and thread-relocation cases described here, including the possible i#3801 IR metadata. Done would require a documented, agreed-upon handling strategy or an implemented fix with coverage for the interruption scenarios.

Written by the indexing model from the issue text.