Various cross-plat support issues in multi-threaded gadget discovery codepaths
- Ngôn ngữ chính
- Python
- Star
- 858
- Fork
- 85
- Merge trung bình
- 7 giờ 40 phút
- Pull request đã merge (30 ngày)
- 1
Mô tả
### Description
Appreciate angrop is not regularly maintained/may never have been intended to run on Windows but heres some issues, included workarounds for fixing these issues purely for visibility (as I am not a massive fan of the solutions...)
## Issue 1 - rop_utils.timeout decorator cross-platform support
Any function decorated with [angrop.rop_utils.timeout](https://github.com/angr/angrop/blob/master/angrop/rop_utils.py#L480) raises `AttributeError: module 'signal' has no attribute 'SIGALRM'` the first time it is called on a Windows host. This is because the `angrop.rop_utils.timeout` decorator uses POSIX only signalling (`signal.SIGALRM`, `signal.alarm`, `signal.setitimer`, etc.) as seen below:
https://github.com/angr/angrop/blob/1ca52e1b48266e611456773df49d9d829bd50e25/angrop/rop_utils.py#L480-L501
Because `GadgetAnalyzer._analyze_gadget`, `ChainBuilder.*`, `MemWriter.*`, and `RopChain.__concretize_chain_values` are all decorated with `@rop_utils.timeout`, the single-threaded gadget finding code path (`rop.find_gadgets_single_threaded`) & chain building both crash on Windows even when the rest of angrop's analysis pipeline should succeed.
### Repro
```python
from angrop.rop_utils import timeout
@timeout(1)
def foo():
return 0xDEAD
if __name__ == "__main__":
# Run on Windows host...
print(foo())
```
### Workaround
NT workaround uses `sys.platform` to guard POSIX-only functionality & implements a fallback daemon watchdog thread which waits for `seconds_before_timeout`. If the wrapped call hasn't returned by then, the watchdog injects `RopTimeoutException` into the calling thread via `ctypes.pythonapi.PyThreadState_SetAsyncExc`. The caller's thread is preserved (no semantic change w.r.t. thread-local state, caches, or logging) & the wrapped function is interrupted (i.e. not just abandoned).
ref: https://github.com/pollingsoon/angrop/commit/eeb9fe8d1783b3dba6d5d14d8714ccbe97a8cba0
```python
if sys.platform != "win32":
# orig POSIX implementation
# ...
else:
# Windows lacks SIGALRM/setitimer. A watchdog thread waits for
# `seconds_before_timeout`; if `f` has not returned by then, the
# watchdog injects `RopTimeoutException` into the calling thread via
# `PyThreadState_SetAsyncExc`. The `__del__` / weakref delay logic
# from the POSIX handler is unnecessary here because async exceptions
# are delivered at bytecode boundaries rather than mid-finalizer?
def _async_raise(thread_ident, exc_type):
"""
Inject `exc_type` into the thread with id `thread_ident`.
Returns True when exactly one thread was affected.
"""
# here be dragons...
# https://docs.python.org/3/c-api/threads.html#c.PyThreadState_SetAsyncExc
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(
ctypes.c_ulong(thread_ident), ctypes.py_object(exc_type)
)
if res > 1:
# More than one thread was affected; revert to avoid corrupting state.
ctypes.pythonapi.PyThreadState_SetAsyncExc(
ctypes.c_ulong(thread_ident), None
)
return res == 1
def timeout(seconds_before_timeout):
def decorate(f):
def new_f(*args, **kwargs):
target_ident = threading.get_ident()
done = threading.Event()
def watchdog():
if done.wait(seconds_before_timeout):
return # f() finished within the budget
_async_raise(target_ident, RopTimeoutException)
t = threading.Thread(target=watchdog, daemon=True)
t.start()
try:
return f(*args, **kwargs)
finally:
done.set()
return new_f
return decorate
```
*Here be dragons*:
GIL-holding code can't be interrupted; `PyThreadState_SetAsyncExc` delivers the exception only at Python bytecode boundaries. A long syscall, e.g. `time.sleep()`, or a C extension that holds the GIL (possibly from within z3, claripy's solver calls or native unicorn execution, etc) will keep running until control returns to the Python interpreter. The timeout fires eventually rather than ASAP. On POSIX platforms, `SIGALRM` interrupts the syscall itself so this results in a suboptimal asymmetry here...
## Issue 2 - gadget_finder worker_func2 cross-platform support
The multiprocessing worker `worker_func2` that drives parallel gadget analysis sets SIGALRM directly as below: https://github.com/angr/angrop/blob/1ca52e1b48266e611456773df49d9d829bd50e25/angrop/gadget_finder/__init__.py#L57 which will raise an `AttributeError: module 'signal' has no attribute 'SIGALRM'` on the NT platform. As such, any call to `rop.find_gadgets(processes>1)` crashes before the first gadget can be analysed when using angrop on Windows.
The single-threaded gadget discovery path (`find_gadgets_single_threaded`) is unaffected.
### Repro
```python
import angr
import angrop # pylint: disable=unused-import
from multiprocessing import cpu_count
if __name__ == "__main__":
# Run on Windows host...
p = angr.Project("libfoo.so", load_options={'main_opts': {'base_addr': 0}})
rop = p.analyses.ROP()
rop.find_gadgets(processes=cpu_count(), optimize=False)
```
### Workaround
NT workaround implements a per-call daemon watchdog thread which waits for `ANALYZE_GADGET_TIMEOUT` and then calls `os._exit(0)` if the analysis has not returned. `mp.Pool` replaces the dead worker as it does on POSIX, so the expected kill worker on timeout behaviour are preserved.
ref: https://github.com/pollingsoon/angrop/commit/de7b3d8981825728688b7d7aec025fed86545f5a
```python
if sys.platform != "win32":
# orig POSIX implementation
# ...
else:
# Windows lacks SIGALRM so replicate the kill the worker on timeout
# behaviour with a per-call watchdog thread that calls `os._exit(0)`
# if the analysis exceeds `ANALYZE_GADGET_TIMEOUT`. `mp.Pool` will
# then replace the dead worker, matching the POSIX behaviour.
def worker_func2(addr, cond_br=None):
analyzer = _global_gadget_analyzer
done = threading.Event()
def watchdog():
if not done.wait(ANALYZE_GADGET_TIMEOUT):
l.warning("[angrop] worker_func2 times out, exit the worker process!")
os._exit(0)
t = threading.Thread(target=watchdog, daemon=True)
t.start()
try:
if cond_br is None:
res = analyzer.analyze_gadget(addr)
else:
res = analyzer.analyze_gadget(addr, allow_conditional_branches=cond_br)
finally:
done.set()
_maybe_exit_for_leak(res)
return _normalize_gadget_result(res)
```
Caveat: each worker spawn re-imports angrop, re-imports angr, and re-runs _set_global_gadget_analyzer, etc. this is significantly slower than POSIX fork CoW behaviour but c'est la vie...
## Issue 3 - find_gadgets fails on Windows due to unpicklable local class SpecialMem (stale PyPI package)
On NT hosts (and likely any platform where `multiprocessing` defaults to `spawn` start method rather than `fork`? e.g. macOS with Python >= 3.8), calling `ROP.find_gadgets(processes>=1)` will crash during `Pool` startup as follows:
`AttributeError: Can't pickle local object 'make_initial_state..SpecialMem'`
followed by a downstream `EOFError: Ran out of input` from a child worker.
The single-threaded gadget discovery path (`find_gadgets_single_threaded`) is unaffected.
N.B. this is already fixed by commit #136 which moves `SpecialMem` definition to module scope allowing `make_initial_state` to call `register_default("sym_memory", SpecialMem)` against the module-level class resulting in qualname of `SpecialMem`, with no ``, thus pickling cleanly.
https://github.com/angr/angrop/blob/1ca52e1b48266e611456773df49d9d829bd50e25/angrop/rop_utils.py#L265-L270
### Repro
```python
import angr
import angrop # pylint: disable=unused-import
from multiprocessing import cpu_count
if __name__ == "__main__":
# Run on Windows host with angrop==9.2.12.post3...
p = angr.Project("libfoo.so", load_options={'main_opts': {'base_addr': 0}})
rop = p.analyses.ROP()
rop.find_gadgets(processes=cpu_count(), optimize=False)
```
### Workaround
The [latest package](https://pypi.org/project/angrop/#history) of angrop available on PyPI is `9.2.12.post3` which predates this fix so it would be ideal if this could be updated/bumped up to `9.2.13.xxx`.
### Steps to reproduce the bug
_No response_
### Environment
## Env
- Python: 3.14
- OS: Windows 11 Pro 10.0.28000 (26H1)
- angrop: 9.2.13.dev0
- claripy: 9.2.213
- angr: 9.2.213
### Additional context
_No response_
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Đánh giá
Issue này chưa được đánh giá.