py-pdf / py-pdf/pypdf

Performance: why is PdfWriter(clone_from=...) much slower than PdfWriter().append_pages_from_reader(PdfReader(...)) ?

Open
#2,136 4 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

nf-documentation nf-performance
Dominant language
Python
Stars
10.2k
Forks
1.6k
Avg merge
1d 6h
Merged PRs (30d)
90

Description

Context: I'm trying to replace my uses of pdfrw by pypdf in various scripts that I have,
and pypdf is still revealing to be quite slower...

In some case, I'm witnessing a factor x5 speed difference between using PdfWriter(clone_from=...) and PdfWriter().append_pages_from_reader(PdfReader(...))

Environment

$ python -m platform
Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.29

$ python -c "import pypdf;print(pypdf._debug_versions)"
pypdf==3.15.4, crypt_provider=('cryptography', '39.0.2'), PIL=9.4.0

Code + PDF

This is a minimal, complete example that shows the issue:

from pypdf import PdfWriter
PdfWriter(clone_from="2020-LaCaravelle_LaMortPrendDesVacances.pdf")

Versus:

from pypdf import PdfReader, PdfWriter
writer = PdfWriter()
writer.append_pages_from_reader(PdfReader("2020-LaCaravelle_LaMortPrendDesVacances.pdf"))

Source PDF file: https://chezsoi.org/lucas/2020-LaCaravelle_LaMortPrendDesVacances.pdf (3.9Mib - 148 pages)

Execution times

$ time ./repro_writer_fast.py $jdr/2020-LaCaravelle_LaMortPrendDesVacances.pdf
real    0m0,819s

$ time ./repro_writer_slow.py $jdr/2020-LaCaravelle_LaMortPrendDesVacances.pdf
real    0m4,551s

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the comparison between PdfWriter(clone_from=...) and PdfWriter().append_pages_from_reader(PdfReader(...)) using the supplied 148-page PDF and the minimal examples. Profile or inspect both entry points to identify the source of the timing difference, then verify that equivalent cloning no longer has the reported performance gap.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.