parseBuffer ignores byteOffset - small PDFs read from the wrong memory
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 2.2k
- Forks
- 394
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 7
Description
I started seeing issues with Invalid XRef stream header while parsing PDF files. They are not loaded from the files but from S3 bucket.
All this happened when there was a rollout of node 24.18 which increased buffer size from 8kB to 64kB (by default).
After long investigation I was able to get full reproduction and understanding of the source of the issue. parseBuffer() assumes the Buffer it is given starts at offset 0 of its underlying ArrayBuffer. Node returns pooled Buffers (views into a shared 64 KB ArrayBuffer) for allocations under 32 KB, and that includes fs.readFileSync() of any file under 32 KB.
Environment
- pdf2json 4.0.3
- Node 24.19.0 (
Buffer.poolSizeis 65536)
Reproduction
npm i pdf2json
node repro.mjs
import fs from "node:fs";
import PDFParser from "pdf2json";
// Write a tiny valid PDF so this script is self-contained.
// Any PDF smaller than 32 KB reproduces it just as well.
const objs = [
"1 0 obj\n<< /Type /Catalog /Pages 2 0 R >>\nendobj\n",
"2 0 obj\n<< /Type /Pages /Kids [3 0 R] /Count 1 >>\nendobj\n",
"3 0 obj\n<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>\nendobj\n",
"4 0 obj\n<< /Length 44 >>\nstream\nBT /F1 12 Tf 20 100 Td (hello pdf2json) Tj ET\nendstream\nendobj\n",
"5 0 obj\n<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>\nendobj\n",
];
let pdf = "%PDF-1.4\n";
const offsets = [];
for (const o of objs) { offsets.push(pdf.length); pdf += o; }
const xref = pdf.length;
pdf += `xref\n0 ${objs.length + 1}\n0000000000 65535 f \n`;
for (const o of offsets) pdf += `${String(o).padStart(10, "0")} 00000 n \n`;
pdf += `trailer\n<< /Size ${objs.length + 1} /Root 1 0 R >>\nstartxref\n${xref}\n%%EOF\n`;
fs.writeFileSync("tiny.pdf", Buffer.from(pdf, "latin1"));
// Uncomment this line and the script passes. It is the only change needed.
// Buffer.poolSize = 0;
// Read the file the ordinary way. No slicing, no views, no manual buffers.
const buffer = fs.readFileSync("tiny.pdf");
console.log("node :", process.version);
console.log("file size :", buffer.length);
console.log("buffer.byteOffset:", buffer.byteOffset);
console.log("buffer.buffer :", buffer.buffer.byteLength, "bytes");
const parser = new PDFParser(null, true);
parser.on("pdfParser_dataReady", () =>
console.log("\nparsed OK:", JSON.stringify(parser.getRawTextContent().split("\n")[0])));
parser.on("pdfParser_dataError", (e) =>
console.log("\nfailed:", String(e?.parserError ?? e)));
parser.parseBuffer(buffer);
Actual output:
node : v24.19.0
file size : 589
buffer.byteOffset: 2400
buffer.buffer : 65536 bytes
failed: Error: Error: Invalid XRef stream header
Uncommenting Buffer.poolSize = 0 is the only change needed to make the same file parse:
buffer.byteOffset: 0
buffer.buffer : 589 bytes
parsed OK: "hello pdf2json"
Probably this is the source of issues
pdfparser.js, in parseBuffer():
let pdfBufferParse = pdfBuffer;
if (pdfBufferParse.buffer.byteLength !== pdfBufferParse.length) {
pdfBufferParse = Buffer.from(pdfBufferParse.buffer, 0, pdfBufferParse.byteLength);
}
When the input is a view into a larger ArrayBuffer, this re-wraps it starting at offset 0 rather than at pdfBuffer.byteOffset, so the parser gets whatever else is in the pool at that position.
Also affects
createParserStream()
Piping the same tiny PDF into createParserStream() fails identically, since it calls parseBuffer(Buffer.concat(chunks)) and Buffer.concat returns a pooled buffer. loadPDF() with a file path is not affected.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in pdfparser.js at parseBuffer(), then trace createParserStream() and its Buffer.concat path. Run the provided repro.mjs with Node 24 and confirm behavior for pooled Buffers with nonzero byteOffset. Done means the tiny PDF parses successfully through parseBuffer() and createParserStream(), while loadPDF() remains unaffected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100