openwall / openwall/john

pdf2john emits empty hash for encrypted PDFs with space-padded xref offsets

Open Beginner friendly
#6,018 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

pdf2john.pl parses classic cross-reference entries with a fixed-width regex:

$buff =~ /^\s*(\d{10}) (\d{5}) (f|n)/s or return -4;

Each entry is 20 bytes, but some encrypted PDFs space-pad the offset/generation instead of zero-padding them:

' 15 00000 n \n' instead of '0000000015 00000 n \n'

These files are malformed per spec, but pdf readers (like pikepdf) accept them, and they appear in the wild as an anti-analysis trick: the PDF still opens in real readers, but pdf2john rejects the table (Warning: Invalid xref table), never reads the /Encrypt dict, and outputs an empty hash (filename: with nothing after it). John then has nothing to crack.

Suggested fix: relax the fixed widths to \d+ - entries are still read in 20-byte units so alignment is unchanged, and output is byte-identical for well-formed PDFs:

$buff =~ /^\s*(\d+)\s+(\d+)\s+(f|n)/s or return -4;
Attaching a minimal reproducer PDF (password: test123) - opens fine in any reader, but yields an empty hash without the fix.
malformed_xref_test.pdf


Malformed encrypted PDF test:

perl run/pdf2john.pl /tmp/malformed_xref_test.pdf
/tmp/malformed_xref_test.pdf:

Intact encrypted PDF test:

perl run/pdf2john.pl /tmp/sample_encrypted_ok.pdf
/tmp/sample_encrypted_ok.pdf:$pdf$5*6*256*-4*1*16*392430db72b0e546a8f6e9c5a5be43ad*48*1326610df39c173efff346e7e38c6e1aff85904a3d603ca332ce83e97bacfc16f8995cc636b178c90455ffd08cc9ca99*48*df6da506e83b1ab003495cfceb3852b725e4d98fd7e8dae3ff350727acac54e720cc502886e2c0ec080b7049e502cb0c*32*c3773142e97f901119f3a7be35d200fb948ee29696ae06529522d85be4fb3116*32*a9908afcfb15c3bb060041bf4b0b4262726526594800d330dc3b817c7368d996

Intact not encrypted PDF test :

perl run/pdf2john.pl /tmp/sample.pdf
/tmp/sample.pdf: not encrypted!

Note: pdf2john.py also fails on this file (raises ValueError: too many values to unpack in xref.py), so it's not a workaround.

uv run --with pyhanko run/pdf2john.py /tmp/malformed_xref_test.pdf
Traceback (most recent call last):
  File "/opt/john/run/pdf2john.py", line 137, in <module>
    extractor = PdfHashExtractor(filename)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/john/run/pdf2john.py", line 67, in __init__
    self.pdf = PdfFileReader(doc, strict=strict)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/uv/archive-v0/465ORgOMVx58ZP64IsgZL/lib/python3.12/site-packages/pyhanko/pdf_utils/reader.py", line 201, in __init__
    self.xrefs, xref_builder = _read_xrefs_and_trailer(stream, self, strict)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/uv/archive-v0/465ORgOMVx58ZP64IsgZL/lib/python3.12/site-packages/pyhanko/pdf_utils/reader.py", line 167, in _read_xrefs_and_trailer
    xref_sections = xref_builder.read_xrefs()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/uv/archive-v0/465ORgOMVx58ZP64IsgZL/lib/python3.12/site-packages/pyhanko/pdf_utils/xref.py", line 778, in read_xrefs
    startxref = self._read_xref_table(declared_startxref)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/uv/archive-v0/465ORgOMVx58ZP64IsgZL/lib/python3.12/site-packages/pyhanko/pdf_utils/xref.py", line 686, in _read_xref_table
    highest = xref_section_data.process_entries(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/uv/archive-v0/465ORgOMVx58ZP64IsgZL/lib/python3.12/site-packages/pyhanko/pdf_utils/xref.py", line 384, in process_entries
    for xref_entry in entries:
  File "/root/.cache/uv/archive-v0/465ORgOMVx58ZP64IsgZL/lib/python3.12/site-packages/pyhanko/pdf_utils/xref.py", line 156, in parse_xref_table
    offset, generation, marker = line[:18].split(b" ")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: too many values to unpack (expected 3)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in run/pdf2john.pl at the fixed-width xref-entry regex described in the issue. Run it against the attached malformed_xref_test.pdf and the intact encrypted sample, then verify that space-padded offsets produce a non-empty hash while well-formed output remains unchanged; also confirm unencrypted PDFs still report that they are not encrypted.

Written by the indexing model from the issue text.

Assessment

Tech stack
perl
Domain
security
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.