trailofbits / trailofbits/polyfile
A w or W string match reports an extent that runs past the end of the file
@ESultanik is already working on this.
Since Sep 18, 2026.
- Dominant language
- Python
- Stars
- 390
- Forks
- 31
- Avg merge
- 7h 52m
- Merged PRs (30d)
- 72
Description
StringMatch.matches reshapes its candidate into a reproduction of libmagic's MAXstring value union before matching, padding with NUL bytes to 128:
data = data[:limit].ljust(MAX_STRING_BYTES, b"\x00")
The padding is right, and it is what makes a w or W blank run stop where libmagic's stops. But the compiled pattern matches against the padded buffer, and MatchedTest takes its length from what the pattern consumed, so a value holding a literal NUL reports bytes that are not in the file:
definition: 0 string/W A\ \0 found
input: b'A ' (3 bytes)
file 5.48: found
PolyFile: found
MatchedTest offset=0 length=4 value='A \x00'
offset + length is 4 on a three-byte file, and value ends in a byte the file does not contain.
The verdict is correct. file matches this too, because the value's NUL meets mcopy's zero-fill. Only the reported extent is wrong.
Why it matters
MatchedTest.length is not just a display field:
explain()reads it asnum_byteswhen it renders hex context (polyfile/magic.py:295-299), so it reads past the buffer.NamedTest.testresolves a child offset fromparent_match.offset + parent_match.length(polyfile/magic.py:3860,3867), so ausenested under such a test starts one byte too far in.
libmagic does not have the problem, because its notion of a =-relation string's extent is m->vallen rather than the bytes consumed (moffset, softmagic.c:904-905), and mprint prints m->value.s rather than the copied buffer (softmagic.c:672-677). PolyFile already models the first for the relative base through declared_length, so only length and value are affected.
Exposure
None today. Of the 139 shipped string and search tests carrying w or W, zero have an escaped NUL in the value. The defect is reachable only from a hand-written or newly imported definition.
Fix
Clamp the match to the bytes that exist, something like:
return self.post_process(bytes(m.group(0))[:len(data)])
against the unpadded length, with a test pinning offset + length <= len(data) for a value containing a NUL.
Provenance
Introduced by #3589, which is otherwise correct and is approved. Filed rather than sent back to the contributor: the padding is the part that needed outside knowledge of mcopy, and this is our own cleanup.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.