DynamoRIO / DynamoRIO/dynamorio

Avoid standard heap in droption and drcachesim init

Open
#4,660 0 comments 0 reactions 0 assignees View on GitHub
Component-DrMemtrace Priority-Low
Dominant language
C
Stars
3.2k
Forks
629
Avg merge
2d 18h
Merged PRs (30d)
30

Description

While typical shared-library usage of DR isolates a client's heap from the app, when DR and clients are statically linked into the application there is unfortunately no way to isolate. We're already using placement new at runtime in drcachesim to support static usage. We'd been assuming that application heap usage at client init time was relatively safe, for internally-triggered attach where the init thread is at a clean ABI point. However, we have hit issues with application heap and tools that replace the allocator with a scheme involving SIGSEGV for detecting bugs. During attach, the init thread then raises SIGSEGV, running into signal delivery problems under #1921. While we plan to solve all the #1921 cases in any case, it may be worth providing app-heap-free options, for possible ptrace-attach with static DR or other cases where worse problems arise, or cases where clients need to parse options post-init-time.

Xref https://github.com/DynamoRIO/dynamorio/issues/4640#issuecomment-755728478 where using std::string with a custom allocator that invokes dr_global_alloc for all droption strings does not work with static DR, due to the static initializers that run natively at process init time. An allocator that looks at dynamo_heap_initialized does not work either, as it doesn't know how to free.

A possible solution is to store only literal char* types at init time and only construct std::string at parse time.

Beyond droption, func_trace has a std::vector and std::set which can easily be made to use a custom allocator.

We may want to make a C++ allocator that uses dr_global_heap part of a library exported by DR. Or bundled with droption.

Contributor guide

Open the contributing guide

Research direction

Start by tracing initialization and option parsing in droption and drcachesim, then inspect func_trace for its std::vector and std::set usage. Compare the proposed literal-char-pointer approach with a DR-backed C++ allocator, including static initialization and freeing. Done means static DR initialization avoids the application heap without breaking parsing or cleanup.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
devtools
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.