Loading an ELF materializes its declared zero-fill, so p_memsz rather than file size decides the cost
- 主要语言
- Python
- 星标
- 485
- 派生
- 135
- 平均合并
- 2 天 2 小时
- 30 天内合并 PR
- 15
描述
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS
Loading an ELF costs `p_memsz`, not file size, because the declared zero-fill is written out as real bytes. A program is free to declare gigabytes of `.bss` in a few hundred kilobytes of file, so an object that is trivial to read can be impossible to load.
There is a fixture in this project's own test corpus that does it:
```python
import cle
cle.Loader("angr/binaries/tests/x86_64/ALLSTAR_aces3_xaces3", auto_load_libs=False)
```
The file is 8.2 MB. One of its load segments declares 6.04 GB of zero-fill:
| segment | `p_filesz` | `p_memsz` | zero-fill |
| --- | ---: | ---: | ---: |
| `PT_LOAD[2]` | 7,262,516 | 7,262,516 | 0 |
| `PT_LOAD[3]` | 394,048 | 6,486,872,008 | **6.04 GB** |
Under a `RLIMIT_AS` of 4 GB that raises `MemoryError`; with no limit it took two OOM kills on a 15.6 GB cgroup here. Measured peak RSS on the way in is **6.13 GB**. Nothing in CI loads this fixture, so it has gone unnoticed.
There are two sites, and the second is only reachable once the first is dealt with:
- `ELF._load_segment` — `data = data.ljust(zeroend - mapstart, b"\0")`
- `ELF.__register_sections` — `b"\0" * sec_readelf.header["sh_size"]` for `SHT_NOBITS`
The obvious repair is to back the zero region with an anonymous `mmap`, which the kernel commits only as pages are touched. I tried it: it works, and peak RSS on that fixture falls from **6.13 GB to 0.107 GB**.
It is not a two-line change, though, and that is why this is a report rather than a pull request. `Clemory.add_backer` accepts an `mmap` and `backers()` is annotated for one, but `__getitem__` and `__setitem__` gate on `isinstance(data, bytearray | list)`, so a byte read from an `mmap` backer raises `KeyError` — which also makes `__contains__` answer `False` and silently defeats the `not in self.memory` guard in `__register_sections`. `get_mmaped_data` returns `bytes` despite the name, so no `mmap` backer exists in practice today and nothing has had to support one. Making the zero region lazy therefore means making `mmap` a first-class backer across several accessors on the read path, which seems like your call rather than mine.
The alternatives I can see both give something up: refusing to load past a size threshold changes behaviour for exactly the binaries this is about, and leaving the region unbacked turns reads that currently return zeroes into `KeyError`.
Happy to open a pull request for whichever direction you prefer.
贡献指南
这个仓库没有索引到贡献指南
评估
这个 Issue 还没有评估数据。