DynamoRIO / DynamoRIO/dynamorio
raw2trace: "memref entry found outside of bb" for generated code in a module's NOBITS section (JIT in .bss)
- Dominant language
- C
- Stars
- 3.2k
- Forks
- 629
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 31
Description
If an app executes JIT/generated code that lives inside a loaded module's address range but in a `NOBITS` section, drmemtrace records no encoding for it. raw2trace then decodes zeros from the on-disk module and aborts the whole thread with memref entry found outside of bb.
**Evidence (aarch64)**
Here, appX is non-PIE, so `link VA == load VA`, and the module base is fixed at `0x200000`.
Maps for the app says it has an anonymous executable region at 0x5b400000 within the mapped app's image:
```
$ cat /proc/$(pidof -s app)/maps | head
00200000-******** r--p 00000000 07:00 789794 /bin/appX
...
5b400000-60900000 rwxp 00000000 00:00 0
********-******** r-xp 58a00000 07:00 789794 /bin/appX
********-******** r-xp 5a200000 07:00 789794 /bin/appX
```
`readelf -SW` for `appX` confirms the anonymous region is a JIT region of type `NOBITS`:
```
Section Headers:
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
...
[65] .app_jit NOBITS 000000005b400000 58957eb0 5500000 00 WA 0 0 2097152
...
```
`readelf -lW /bin/appX` shows that the RW LOAD segment reserves far more memory than it file-backs, and that MemSiz covers the JIT region, which is why DR attributes it to the module:
```
Type Offset VirtAddr FileSiz MemSiz Flg Align
LOAD 0x57f58000 0x0000000058158000 0x9ffeb0 0x87a8000 RW 0x200000
```
MemSiz end = 0x58158000 + 0x87a8000 = 0x60900000, which covers .app_jit [0x5b400000, 0x60900000); file backing stops at 0x58158000 + 0x9ffeb0 = 0x58b57eb0.
**drraw2trace** correctly fails on trying to decode a JIT address without an encoding and with `-verbose 3` prints this out:
```
[drmemtrace]: Appending 4 instrs in bb 0xfffbedae010c in mod 3 +0x5e5e010c = /bin/appX
[drmemtrace]: extra memref entry: 0xffffe4493fd0
[drmemtrace]: Worker 0 hit error Failed to process file for thread 365750: memref entry found outside of bb on trace thread 0
ERROR: Conversion failed: Failed to process file for thread 365750: memref entry found outside of bb
```
Offset `0x5e5e010c` for module `/bin/appX` (mod 3, base: 0x00200000) is VA `0x5e7e010c` matching the NOBITS JIT region above and it won't be in the module binary.
Only the JIT thread fails; other threads of the same process, using the same modules.log and encodings.bin, convert fine.
**Possible root cause**: [`does_pc_require_encoding()`](https://github.com/DynamoRIO/dynamorio/blob/4fe110619bdcc98bb41f44b94f89b79b5ff5f940/clients/drcachesim/tracer/instru_offline.cpp#L554) records an encoding only when drmodtrack_lookup misses or it's the vDSO. In-module NOBITS code gets a valid `modidx`, so nothing is recorded.
Contributor guide
Research direction
Start in clients/drcachesim/tracer/instru_offline.cpp at does_pc_require_encoding(), then trace how modules.log and encodings.bin are consumed by raw2trace. Reproduce the aarch64 JIT-in-NOBITS case described in the report and verify that conversion no longer aborts on the in-module address while ordinary module decoding remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, linux
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100