DynamoRIO / DynamoRIO/drmemory

tailcall from alloc routine A to alloc routine B breaks DrMemory assumptions

Open
#128 0 comments 0 reactions 0 assignees View on GitHub
Migrated Priority-Low
Dominant language
C
Stars
2.7k
Forks
290
PR merge metrics
No merged PRs in 30d

Description

_From [derek.br...@gmail.com](https://code.google.com/u/117968039472581148324/) on December 10, 2010 17:57:52_

PR 418138

We assume that each call site calls at most one alloc routine.
This can be violated by indirect calls that vary or by a direct
call to A that then tailcalls B. The latter actually happens
with realloc(NULL,) on Fedora10's glibc tailcalling malloc.
Our recursion count gets off and the rest of the run is messed up.
Note that using rets in callee as post instru point doesn't help: tailcall won't
hit the ret in outer callee.
Not clear how to solve.
- rather than giving up and going to our own malloc routines, I'm relying
on the identification of tailcall jmps (which we already have in place,
though not guaranteed of course, if an indirect jmp) to handle
tailcalls. it works for the fedora10 scenario.

_Original issue: http://code.google.com/p/drmemory/issues/detail?id=128_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.