trailofbits / trailofbits/polyfile

A w or W string match reports an extent that runs past the end of the file

Open
#3,600 0 comments 0 reactions 1 assignee View on GitHub

@ESultanik is already working on this.

Since Sep 18, 2026.

bug
Dominant language
Python
Stars
390
Forks
31
Avg merge
7h 52m
Merged PRs (30d)
72

Description

StringMatch.matches reshapes its candidate into a reproduction of libmagic's MAXstring value union before matching, padding with NUL bytes to 128:

data = data[:limit].ljust(MAX_STRING_BYTES, b"\x00")

The padding is right, and it is what makes a w or W blank run stop where libmagic's stops. But the compiled pattern matches against the padded buffer, and MatchedTest takes its length from what the pattern consumed, so a value holding a literal NUL reports bytes that are not in the file:

definition: 0	string/W	A\ \0	found
input:      b'A  '   (3 bytes)

file 5.48:  found
PolyFile:   found
            MatchedTest offset=0 length=4 value='A  \x00'

offset + length is 4 on a three-byte file, and value ends in a byte the file does not contain.

The verdict is correct. file matches this too, because the value's NUL meets mcopy's zero-fill. Only the reported extent is wrong.

Why it matters

MatchedTest.length is not just a display field:

  • explain() reads it as num_bytes when it renders hex context (polyfile/magic.py:295-299), so it reads past the buffer.
  • NamedTest.test resolves a child offset from parent_match.offset + parent_match.length (polyfile/magic.py:3860, 3867), so a use nested under such a test starts one byte too far in.

libmagic does not have the problem, because its notion of a =-relation string's extent is m->vallen rather than the bytes consumed (moffset, softmagic.c:904-905), and mprint prints m->value.s rather than the copied buffer (softmagic.c:672-677). PolyFile already models the first for the relative base through declared_length, so only length and value are affected.

Exposure

None today. Of the 139 shipped string and search tests carrying w or W, zero have an escaped NUL in the value. The defect is reachable only from a hand-written or newly imported definition.

Fix

Clamp the match to the bytes that exist, something like:

return self.post_process(bytes(m.group(0))[:len(data)])

against the unpadded length, with a test pinning offset + length <= len(data) for a value containing a NUL.

Provenance

Introduced by #3589, which is otherwise correct and is approved. Filed rather than sent back to the contributor: the padding is the part that needed outside knowledge of mcopy, and this is our own cleanup.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.