Performance: why is PdfWriter(clone_from=...) much slower than PdfWriter().append_pages_from_reader(PdfReader(...)) ?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 10.2k
- Forks
- 1.6k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 90
Description
Context: I'm trying to replace my uses of pdfrw by pypdf in various scripts that I have,
and pypdf is still revealing to be quite slower...
In some case, I'm witnessing a factor x5 speed difference between using PdfWriter(clone_from=...) and PdfWriter().append_pages_from_reader(PdfReader(...))
Environment
$ python -m platform
Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.29
$ python -c "import pypdf;print(pypdf._debug_versions)"
pypdf==3.15.4, crypt_provider=('cryptography', '39.0.2'), PIL=9.4.0
Code + PDF
This is a minimal, complete example that shows the issue:
from pypdf import PdfWriter
PdfWriter(clone_from="2020-LaCaravelle_LaMortPrendDesVacances.pdf")
Versus:
from pypdf import PdfReader, PdfWriter
writer = PdfWriter()
writer.append_pages_from_reader(PdfReader("2020-LaCaravelle_LaMortPrendDesVacances.pdf"))
Source PDF file: https://chezsoi.org/lucas/2020-LaCaravelle_LaMortPrendDesVacances.pdf (3.9Mib - 148 pages)
Execution times
$ time ./repro_writer_fast.py $jdr/2020-LaCaravelle_LaMortPrendDesVacances.pdf
real 0m0,819s
$ time ./repro_writer_slow.py $jdr/2020-LaCaravelle_LaMortPrendDesVacances.pdf
real 0m4,551s
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the comparison between PdfWriter(clone_from=...) and PdfWriter().append_pages_from_reader(PdfReader(...)) using the supplied 148-page PDF and the minimal examples. Profile or inspect both entry points to identify the source of the timing difference, then verify that equivalent cloning no longer has the reported performance gap.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100