microsoft / microsoft/markitdown
PDF performance (PDFMiner)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 186k
- Forks
- 13.7k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 49
Description
Hi there
I've been using MarkItDown for conversion of some PDF files in a project I'm working on and I've noticed it performs really poorly with larger documents.
This is in stark contrast to other libraries like PyMuPDF and specifically its markdown variant (PyMuPDF4LLM) which in my tests performed much faster.
Below is a sample test:
- 1.6MB PDF File
- 122 Pages of Text
- Processing Times by Library:
- MarkItDown (PDFMiner): 33s
- PyMyPDF4LLM: 9.24s
This is a huge difference (I believe PDFMiner is completely synchronous so performance falls off significantly the larger a file is) and from what I've seen from others' testing PDFMiner's quality is pretty average with other libraries, unless the team has observed otherwise?
I'm just opening this issue to see what the reasoning was behind using PDFMiner for MarkItDown vs other libraries?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the PDF conversion path that uses PDFMiner and reproduce the reported 1.6MB, 122-page benchmark. Compare processing time and output quality with the referenced PyMuPDF4LLM behavior; done means an agreed direction for the PDF backend, supported by benchmark results and tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100