DynamoRIO / DynamoRIO/dynamorio

native_exec: regain control when calling from a native module into a non-native module

Open
#978 5 comments 0 reactions 0 assignees View on GitHub
Migrated Priority-Medium Type-Feature
Dominant language
C
Stars
3.2k
Forks
629
Avg merge
2d 15h
Merged PRs (30d)
31

Description

_From [rnk@google.com](https://code.google.com/u/rnk@google.com/) on November 14, 2012 16:36:32_

For hybrid instrumentation tools, the dream is to be able to execute the main module natively and dynamically instrument everything else. This should drastically improve startup time for such tools.

Our current -native_exec support is all about calling out to a module that will run natively, and then regaining control when it returns or from our syscall hooks. This issue covers extending it to support regaining control when the native module calls to code that we want to instrument.

There are three ways we can regain control:
- Patch the PLT for direct calls
- NX protect the non-native modules
- Have the compiler annotate indirect calls in the native code

In the PLT, we can basically create a little thunk that takes control. There are details to work out, especially with delay-load imports on Windows and the equivalent RTLD_LAZY on Linux. Ideally, we can find a way to fit the thunk into the existing PLT code sequences, and then we don't have to allocate memory for stubs.

The NX protection is nice because it provides perfect error checking for us, but I'd rather avoid faults if possible. It makes debugging harder, and could have bad performance in a tight loop.

I've actually implemented the LLVM side of the annotations in msan, although I haven't sent it for review. Mostly I just wanted to verify that it is doable. We can do this last once we've got the initial re-takeover support.

For the actual transition point, I'm thinking we want to find a way to quickly jump into an optimized IBL without spilling the full mcontext or switching stacks. We'll also have to swap the library segment.

_Original issue: http://code.google.com/p/dynamorio/issues/detail?id=978_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.