Gallopsled / Gallopsled/pwntools

Update LIBCDB offsets, support real lookup via DynELF, and various enhancements to ELF to support add'l architectures

Open
#1,910 0 comments 0 reactions 0 assignees View on GitHub
feature
Dominant language
Python
Stars
13.7k
Forks
1.9k
Avg merge
6d 23h
Merged PRs (30d)
3

Description

I ran a script over all of https://gitlab.com/libcdb/libcdb to regenerate the build_id offsets that we have stored in Pwnlib. I also took the opportunity to ensure that the offsets are in frequency order (most common is first offset).

The list is reproduced below, but I don't have time to commit right now. Note that some of the EM_ARCH values are not accounted for in `ELF.get_machine_arch` and should also be added so that `ELF.arch` is more accurate.

These values were derived from 6607 unique libc binaries. We might want to add lots of CTF challenge binaries to a corpus somewhere so those are included as well.

**EDIT**: This is wrong and it seems the best-offsets are sorted backwards for some reason.

```
{
'aarch64': [0x248,0x280,],
'amd64': [0x328,0x3b0,0x1c4,0x390,0x2f0,0x1e4,0x380,0x184,0x280,],
'arm': [0x184,],
'em_68k': [0x184,],
'em_parisc': [0x144,0x164,0x184,],
'em_riscv': [0x248,0x280,],
'em_s390': [0x280,0x184,],
'em_sh': [0x184,],
'i386': [0x1c4,0x1e4,0x184,],
'ia64': [0x210,0x248,],
'mips': [0xe868,0xeea8,0xe570,0xebd8,0xe1b0,0xe1d8,0xe548,0xe688,0xecc8,0xed68,0xec28,0xeca0,0xed18,0xe7c8,0xed40,0xe8b8,0x1dc,0x1e0,0x218,],
'mips64': [0xeef8,0xec78,0xede0,0xf588,0xf560,0xee08,0xeed0,0xebb0,0xec00,0xec28,0xee58,0xee80,0x218,],
'mipsel': [0x280,],
'powerpc': [0x184,],
'powerpc64': [0x280,0x248,],
'sparc': [0x184,],
'sparc64': [0x280,],
}
```

Frequency sorting was needed as certain offsets occur much more frequently than others. Take `amd64` and `aarch64` for example.

**EDIT**: It looks like my output above is incorrect for frequency-sorting. I've included the full output of the script in a `` below.

```
{'aarch64': defaultdict(.. at 0x108f86160>,
{584: 1,
640: 20}),
...
'amd64': defaultdict(.. at 0x108602430>,
{388: 197,
452: 4,
484: 17,
640: 546,
752: 7,
808: 2,
896: 29,
912: 7,
944: 4}),
```

Full Output of Offset-to-Count

```
defaultdict( at 0x108069160>,
{'aarch64': defaultdict(.. at 0x108f86160>,
{584: 1,
640: 20}),
'amd64': defaultdict(.. at 0x108602430>,
{388: 197,
452: 4,
484: 17,
640: 546,
752: 7,
808: 2,
896: 29,
912: 7,
944: 4}),
'arm': defaultdict(.. at 0x108f86940>,
{388: 138}),
'em_68k': defaultdict(.. at 0x108f86040>,
{388: 15}),
'em_parisc': defaultdict(.. at 0x108f86a60>,
{324: 1,
356: 1,
388: 2}),
'em_riscv': defaultdict(.. at 0x109102790>,
{584: 1,
640: 1}),
'em_sh': defaultdict(.. at 0x109f570d0>,
{388: 8}),
'i386': defaultdict(.. at 0x109487b80>,
{388: 1424,
452: 4,
484: 17}),
'ia64': defaultdict(.. at 0x108f86f70>,
{528: 1,
584: 95}),
'mips': defaultdict(.. at 0x108f86d30>,
{476: 75,
480: 108,
536: 131,
57776: 2,
57816: 2,
58696: 4,
58736: 1,
59016: 6,
59336: 19,
59496: 1,
59576: 44,
60376: 2,
60456: 8,
60576: 9,
60616: 7,
60696: 11,
60736: 20,
60776: 7,
61096: 1}),
'mips64': defaultdict(.. at 0x109487790>,
{536: 45,
60336: 3,
60416: 4,
60456: 5,
60536: 1,
60896: 2,
60936: 2,
61016: 5,
61056: 15,
61136: 3,
61176: 1,
62816: 2,
62856: 2}),
'mipsel': defaultdict(.. at 0x109f57280>,
{640: 187}),
'powerpc': defaultdict(.. at 0x108e94310>,
{388: 273}),
'powerpc64': defaultdict(.. at 0x109f57160>,
{584: 250,
640: 43}),
's390': defaultdict(.. at 0x1086630d0>,
{388: 190,
640: 124}),
'sparc': defaultdict(.. at 0x109f57820>,
{388: 125}),
'sparc64': defaultdict(.. at 0x108ee5430>,
{640: 177})})
```

We should update the offsets to these, and add additional logic to DynELF to perform dynamic lookup `.note.gnu.build-id` for those architectures with a lot of offsets, or if all of the offsets are wrong. Each offset lookup requires dumping ~16 bytes, so at a certain point (e.g. for `mips` with MANY offsets) it might be optimal to perform the ACTUAL lookup in a "proper" way.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.