angr / angr/angrop

Various cross-plat support issues in multi-threaded gadget discovery codepaths

Đang mở
#155 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
bug
Ngôn ngữ chính
Python
Star
858
Fork
85
Merge trung bình
7 giờ 40 phút
Pull request đã merge (30 ngày)
1

Mô tả

### Description

Appreciate angrop is not regularly maintained/may never have been intended to run on Windows but heres some issues, included workarounds for fixing these issues purely for visibility (as I am not a massive fan of the solutions...)

## Issue 1 - rop_utils.timeout decorator cross-platform support

Any function decorated with [angrop.rop_utils.timeout](https://github.com/angr/angrop/blob/master/angrop/rop_utils.py#L480) raises `AttributeError: module 'signal' has no attribute 'SIGALRM'` the first time it is called on a Windows host. This is because the `angrop.rop_utils.timeout` decorator uses POSIX only signalling (`signal.SIGALRM`, `signal.alarm`, `signal.setitimer`, etc.) as seen below:

https://github.com/angr/angrop/blob/1ca52e1b48266e611456773df49d9d829bd50e25/angrop/rop_utils.py#L480-L501

Because `GadgetAnalyzer._analyze_gadget`, `ChainBuilder.*`, `MemWriter.*`, and `RopChain.__concretize_chain_values` are all decorated with `@rop_utils.timeout`, the single-threaded gadget finding code path (`rop.find_gadgets_single_threaded`) & chain building both crash on Windows even when the rest of angrop's analysis pipeline should succeed.

### Repro

```python
from angrop.rop_utils import timeout

@timeout(1)
def foo():
return 0xDEAD

if __name__ == "__main__":
# Run on Windows host...
print(foo())
```

### Workaround

NT workaround uses `sys.platform` to guard POSIX-only functionality & implements a fallback daemon watchdog thread which waits for `seconds_before_timeout`. If the wrapped call hasn't returned by then, the watchdog injects `RopTimeoutException` into the calling thread via `ctypes.pythonapi.PyThreadState_SetAsyncExc`. The caller's thread is preserved (no semantic change w.r.t. thread-local state, caches, or logging) & the wrapped function is interrupted (i.e. not just abandoned).

ref: https://github.com/pollingsoon/angrop/commit/eeb9fe8d1783b3dba6d5d14d8714ccbe97a8cba0
```python
if sys.platform != "win32":
# orig POSIX implementation
# ...

else:
# Windows lacks SIGALRM/setitimer. A watchdog thread waits for
# `seconds_before_timeout`; if `f` has not returned by then, the
# watchdog injects `RopTimeoutException` into the calling thread via
# `PyThreadState_SetAsyncExc`. The `__del__` / weakref delay logic
# from the POSIX handler is unnecessary here because async exceptions
# are delivered at bytecode boundaries rather than mid-finalizer?
def _async_raise(thread_ident, exc_type):
"""
Inject `exc_type` into the thread with id `thread_ident`.
Returns True when exactly one thread was affected.
"""
# here be dragons...
# https://docs.python.org/3/c-api/threads.html#c.PyThreadState_SetAsyncExc
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(
ctypes.c_ulong(thread_ident), ctypes.py_object(exc_type)
)
if res > 1:
# More than one thread was affected; revert to avoid corrupting state.
ctypes.pythonapi.PyThreadState_SetAsyncExc(
ctypes.c_ulong(thread_ident), None
)
return res == 1

def timeout(seconds_before_timeout):
def decorate(f):
def new_f(*args, **kwargs):
target_ident = threading.get_ident()
done = threading.Event()

def watchdog():
if done.wait(seconds_before_timeout):
return # f() finished within the budget
_async_raise(target_ident, RopTimeoutException)

t = threading.Thread(target=watchdog, daemon=True)
t.start()
try:
return f(*args, **kwargs)
finally:
done.set()
return new_f
return decorate
```

*Here be dragons*:
GIL-holding code can't be interrupted; `PyThreadState_SetAsyncExc` delivers the exception only at Python bytecode boundaries. A long syscall, e.g. `time.sleep()`, or a C extension that holds the GIL (possibly from within z3, claripy's solver calls or native unicorn execution, etc) will keep running until control returns to the Python interpreter. The timeout fires eventually rather than ASAP. On POSIX platforms, `SIGALRM` interrupts the syscall itself so this results in a suboptimal asymmetry here...

## Issue 2 - gadget_finder worker_func2 cross-platform support

The multiprocessing worker `worker_func2` that drives parallel gadget analysis sets SIGALRM directly as below: https://github.com/angr/angrop/blob/1ca52e1b48266e611456773df49d9d829bd50e25/angrop/gadget_finder/__init__.py#L57 which will raise an `AttributeError: module 'signal' has no attribute 'SIGALRM'` on the NT platform. As such, any call to `rop.find_gadgets(processes>1)` crashes before the first gadget can be analysed when using angrop on Windows.

The single-threaded gadget discovery path (`find_gadgets_single_threaded`) is unaffected.

### Repro

```python
import angr
import angrop # pylint: disable=unused-import
from multiprocessing import cpu_count

if __name__ == "__main__":
# Run on Windows host...
p = angr.Project("libfoo.so", load_options={'main_opts': {'base_addr': 0}})
rop = p.analyses.ROP()
rop.find_gadgets(processes=cpu_count(), optimize=False)
```

### Workaround

NT workaround implements a per-call daemon watchdog thread which waits for `ANALYZE_GADGET_TIMEOUT` and then calls `os._exit(0)` if the analysis has not returned. `mp.Pool` replaces the dead worker as it does on POSIX, so the expected kill worker on timeout behaviour are preserved.

ref: https://github.com/pollingsoon/angrop/commit/de7b3d8981825728688b7d7aec025fed86545f5a
```python
if sys.platform != "win32":
# orig POSIX implementation
# ...

else:
# Windows lacks SIGALRM so replicate the kill the worker on timeout
# behaviour with a per-call watchdog thread that calls `os._exit(0)`
# if the analysis exceeds `ANALYZE_GADGET_TIMEOUT`. `mp.Pool` will
# then replace the dead worker, matching the POSIX behaviour.
def worker_func2(addr, cond_br=None):
analyzer = _global_gadget_analyzer
done = threading.Event()

def watchdog():
if not done.wait(ANALYZE_GADGET_TIMEOUT):
l.warning("[angrop] worker_func2 times out, exit the worker process!")
os._exit(0)

t = threading.Thread(target=watchdog, daemon=True)
t.start()
try:
if cond_br is None:
res = analyzer.analyze_gadget(addr)
else:
res = analyzer.analyze_gadget(addr, allow_conditional_branches=cond_br)
finally:
done.set()

_maybe_exit_for_leak(res)
return _normalize_gadget_result(res)
```

Caveat: each worker spawn re-imports angrop, re-imports angr, and re-runs _set_global_gadget_analyzer, etc. this is significantly slower than POSIX fork CoW behaviour but c'est la vie...

## Issue 3 - find_gadgets fails on Windows due to unpicklable local class SpecialMem (stale PyPI package)
On NT hosts (and likely any platform where `multiprocessing` defaults to `spawn` start method rather than `fork`? e.g. macOS with Python >= 3.8), calling `ROP.find_gadgets(processes>=1)` will crash during `Pool` startup as follows:
`AttributeError: Can't pickle local object 'make_initial_state..SpecialMem'`
followed by a downstream `EOFError: Ran out of input` from a child worker.

The single-threaded gadget discovery path (`find_gadgets_single_threaded`) is unaffected.

N.B. this is already fixed by commit #136 which moves `SpecialMem` definition to module scope allowing `make_initial_state` to call `register_default("sym_memory", SpecialMem)` against the module-level class resulting in qualname of `SpecialMem`, with no ``, thus pickling cleanly.

https://github.com/angr/angrop/blob/1ca52e1b48266e611456773df49d9d829bd50e25/angrop/rop_utils.py#L265-L270

### Repro
```python
import angr
import angrop # pylint: disable=unused-import
from multiprocessing import cpu_count

if __name__ == "__main__":
# Run on Windows host with angrop==9.2.12.post3...
p = angr.Project("libfoo.so", load_options={'main_opts': {'base_addr': 0}})
rop = p.analyses.ROP()
rop.find_gadgets(processes=cpu_count(), optimize=False)
```

### Workaround

The [latest package](https://pypi.org/project/angrop/#history) of angrop available on PyPI is `9.2.12.post3` which predates this fix so it would be ideal if this could be updated/bumped up to `9.2.13.xxx`.

### Steps to reproduce the bug

_No response_

### Environment

## Env
- Python: 3.14
- OS: Windows 11 Pro 10.0.28000 (26H1)
- angrop: 9.2.13.dev0
- claripy: 9.2.213
- angr: 9.2.213

### Additional context

_No response_

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.