trailofbits / trailofbits/polyfile
A level 0 message reached through use or indirect gets a space where libmagic emits a separator
@ESultanik is already working on this.
Since Sep 18, 2026.
- Dominant language
- Python
- Stars
- 390
- Forks
- 31
- Avg merge
- 7h 52m
- Merged PRs (30d)
- 72
Description
This supersedes the diagnosis this issue was filed with, and subsumes #3604. The defect is not about \b. PolyFile applies libmagic's continuation spacing rule to every message, including the level 0 messages that a use or an indirect pulls in, where libmagic uses a different rule entirely.
What libmagic does
Two separate rules, in file/src/softmagic.c:
A level 0 entry prints print_sep(ms, *firstline) before its description, never a space (softmagic.c:319-332):
if (*m->desc) {
*found_match = 1;
if (print) {
*returnval = 1;
*need_separator = 1;
*printed_something = 1;
if (print_sep(ms, *firstline) == -1)
return -1;
if (mprint(ms, m) == -1)
return -1;
}
}
print_sep emits nothing while firstline is set, and FILE_SEPARATOR — "\n- " — once it is clear (softmagic.c:2595-2604).
A continuation prints a space, if something printed before it and the entry does not carry NOSPACE (softmagic.c:462-469):
if (*need_separator && (m->flag & NOSPACE) == 0) {
if (file_printf(ms, " ") == -1)
return -1;
}
NOSPACE is set when the description begins with \b (apprentice.c:2428-2435).
firstline clears at the end of any match() that printed (softmagic.c:492-500), and the pointer is shared with the recursions that use (softmagic.c:2027) and indirect (softmagic.c:1968-1973) spawn. So a nested run that printed leaves the next level 0 message to be separated by "\n- ".
Evidence
Each line is file 5.48 built from the submodule, against a definition written for the case:
| case | file |
PolyFile |
|---|---|---|
use carrying \b |
baseINNER |
base INNER |
use with no \b |
baseINNER |
base INNER |
\b on a continuation's own message |
basenext |
basenext |
one indirect |
base:first-nested |
base: first-nested |
two indirects |
base:first-nested\012- second-nested |
base: first-nested second-nested |
The second row is what rules out \b: a use with no \b at all still gets no space. The last row is what shows the separator, and shows it appearing before the second nested verdict rather than the first — the first re-entry printed, which cleared firstline.
That also explains a Mach-O universal binary, where \012- appears three times:
file: Mach-O universal binary with 2 architectures: [x86_64:\012- Mach-O 64-bit x86_64 executable, flags:<...>] [\012- arm64e (caps: 0x2):\012- Mach-O 64-bit arm64e (caps: PAC00) executable, flags:<...>]
PolyFile: Mach-O universal binary with 2 architectures: [ x86_64: Mach-O 64-bit x86_64 executable, flags:<...>] [ arm64e (caps: 0x2): Mach-O 64-bit arm64e (caps: PAC00) executable, flags:<...>]
The space in : [ is not a separator: magic_defs/cafebabe:21 declares \b [, so the description itself is [.
Where it is
Match._soft_magic_message inserts a space before any message that does not begin with \b, whatever level the test sits at, and never emits MATCH_SEPARATOR within a match.
Each result knows its level through result.test.level, so the level 0 rule is reachable. Modeling firstline needs the recursion boundaries that use and indirect introduce, which the flat result sequence does not record directly, though TestResult.parent does carry the tree.
Why it matters
With #3602 merged, a universal binary's nested descriptions hold the right content and still do not match file byte for byte, for this reason alone. On a 2,000-file sample that is about 40 files, the largest single cluster of strongest-match disagreements.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.